This repository contains a list of example projects for the AWS DataOps Development Kit (DDK).
This repository holds our official list of DDK examples code. They aim to showcase the different ways you can leverage the DDK to help with your DataOps workloads, including building a data pipeline to ingest SaaS data or to create your private code artifactory, for example.
We welcome contributions to this repository in the form of enhancements to existing examples or the addition of new ones. For more information on contributing, please see the CONTRIBUTING guide.
It is assumed that you are already somewhat familiar with the AWS DDK. If not, we strongly recommend that you go through our Learning Resources first, in particular the DDK workshop and documentation.
If you would like to start your journey by looking at an example, we recommend you to start with Kinesis to S3 Data Pipeline example from the list below.
Create this optional directory for DDK patterns
mkdir directory_name
cd directory_name
Download the helper script using the below command
curl -LJO https://raw.githubusercontent.com/aws-samples/aws-ddk-examples/main/cli_helper.py
Once the script is downloaded, execute the script using the below command
pip install urllib3
For help regarding usage
python3 cli_helper.py -h
For list existing patterns
python3 cli_helper.py -t "list"
To init an available pattern into your directory
python3 cli_helper.py -t "init" -p "sdlf-ddk-lightweight" -l "python"
Earlier Versions of DDK See here for examples using the library before the
1.0.0
major version release.
Beyond this repository, there are other resources that can be referenced to assist with your learning/development process.
This library is licensed under the Apache 2.0 License.