Search Results: etl-pipelines

Found 19 Skills

Data Processingaws/agent-toolkit-for-aws

ingesting-into-data-lake

Import data into the AWS data lake from S3 files, local uploads, JDBC databases (Oracle, SQL Server, PostgreSQL, MySQL, RDS, Aurora), Amazon Redshift, Snowflake, BigQuery, DynamoDB, or existing Glue catalog tables (migration). Default target is S3 Tables; standard Iceberg on a general purpose bucket is supported where S3 Tables is not adopted. Handles one-time loads, recurring pipelines, migrations. Triggers on: import data, load data, ingest, sync database, migrate table, move data to AWS, set up pipeline, ETL, pull from Snowflake, query BigQuery into S3, export DynamoDB, CTAS, convert to Iceberg. Do NOT use for setting up or troubleshooting Glue connections (use connecting-to-data-source), creating empty tables (use creating-data-lake-table), running queries (use querying-data-lake), finding tables by fuzzy name (use finding-data-lake-assets), catalog audit (use exploring-data-catalog), or SaaS platforms like Salesforce, ServiceNow, SAP, MongoDB, Kafka.

🇺🇸|EnglishTranslated

Data Processingaradotso/data-skills

harvard-art-museum-etl-analytics

Build end-to-end ETL pipelines and analytics dashboards using Harvard Art Museums API data with Python, SQL, and Streamlit

🇺🇸|EnglishTranslated

Data Processingaradotso/data-skills

harvard-artifacts-etl-analytics

Build ETL pipelines and analytics dashboards using Harvard Art Museums API with SQL and Streamlit

🇺🇸|EnglishTranslated

Data Processingaradotso/data-skills

harvard-artifacts-collection-etl-analytics

Build end-to-end ETL pipelines and analytics dashboards using the Harvard Art Museums API with Python, SQL, and Streamlit

🇺🇸|EnglishTranslated

Data Processingaradotso/data-skills

harvard-artifacts-etl-pipeline

Build ETL pipelines and analytics dashboards for Harvard Art Museums API data with MySQL storage and Streamlit visualization

🇺🇸|EnglishTranslated

Data Processingmicrosoftdocs/agent-skill...

azure-data-factory

Expert knowledge for Azure Data Factory development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use when designing ADF pipelines, mapping data flows, SHIR/SSIS IR, SAP CDC, or CI/CD with ARM/DevOps, and other Azure Data Factory related development tasks. Not for Azure Synapse Analytics (use azure-synapse-analytics), Azure Databricks (use azure-databricks), Azure Stream Analytics (use azure-stream-analytics), Azure Data Explorer (use azure-data-explorer).

🇺🇸|EnglishTranslated

Data Processingaradotso/data-skills

amee-joshi-data-engineering-portfolio

Reference portfolio demonstrating Azure data engineering patterns, Medallion architecture, and end-to-end analytics solutions

🇺🇸|EnglishTranslated