Pipeline Jobs
Pipeline Jobs
Detailed description of what a job is in Daton
A job in Daton is defined as one scheduled execution of an extract+load for a table/API. A source integration may have been configured to replicate one or more tables in the data warehouse. Data extraction and warehouse load for each table is considered a job.
Jobs are scheduled as per the Frequency selected while creating an integration.
Every data load or data replication job that Daton runs is logged in the system with detailed information regarding the job. There are a few sections in Daton that list job details for the users.
My Integrations Page - Previous Job Status
My Integration page displays the most recent job status for an integration. The status is displayed in three color codes -
- Green - All jobs for integration in the previous run are successful without any errors
- Orange - Not all jobs for integration in the previous run are successful. Job for at least one API/table has failed
- Red - All jobs for integration in the previous run have failed
Integration Details Page:
Clicking on the icon on my integration page or by clicking on the integration name will take the user to the details of a specific integration. This page gives a user more detailed information pertaining to a specific source integration.
The following job details are displayed here -
- Job Id - Internal ID of the job
- Table Name - Warehouse table name
- Job Start Time - The time at which data extraction is initiated
- Job End Time - The time at which data is loaded into the warehouse
- Job Status
- Rows processed - Number of records written to the warehouse in a particular job
- Time Taken - Total time taken by the job in minutes