dlt - An open-source library added in Python scripts to load data from various and often messy data sources into well-structured, live datasets.
pip install dlt
python pipeline_file_name.py
This is the name of your python file.
dlt pipeline pipeline_name show
This is the name of your pipeline defined in your python file.
Example : The pipeline african_countries
loads data from a defined source
to DuckDB
.
# Create a pipeline.
pipeline = dlt.pipeline(
pipeline_name="african_countries",
destination="duckdb",
dataset_name="countries" # Database schema "countries".
)