What’s New in Daton?

This section includes:

Platform and Connector Enhancements: Find out how we have optimized your process flows, fixed the bugs that irritated you, and advanced the performance of the most loved connectors.

New Connectors: Our library is growing by leaps and bounds every day; see the ones recently added that will allow you to unite Daton with an even greater number of data sources.

Connector Enhancements

Amazon Sponsored Products

Change made to all table ending with "reports"

• Users were facing 'report generation failed' errors every time we encounter internal server errors. We have improved on our report generation process to retry and handle internal servers better.
• This fix will ensure that our report tables continue to fetch the data on time without any interruptions.
• No action is required from the users.

Change applicable for all raw tables

• Our pipelines monitoring alerts found the long existing issues of raw tables having long running jobs. We have updated our pagination mechanism, that has reduced the occurrence of long running jobs by 50%.
•  This improvement will faster the data processing and make our system more efficient.
• No action is required from users.

Amazon Sponsored Display and Sponsored Brands

Table Name: SponsoredBrands_BudgetRulesRecommendation

• BudgetRulesRecommendation table has had issues with some Campaign IDs that caused bad request - unsupported marketplace error.
• We have filtered out the Campaign IDs to avoid getting into bad request errors.
• This fix will ensure that BudgetRulesRecommendation table continue to fetch the data on time without any interruptions.
• No action is required from users.

Change applicable for all the raw tables

• Error messages shown on the UI were not user-friendly.
• We have updated our error handling scenarios, made error message more readable in case of server issues.
• This improvement will allow users to understand the root cause of their failed table and ensure transparency and easy troubleshooting.
• No action is required from users.

Amazon Selling Partner

Table Name: FBA Returns Report

• We encountered multiple records with the same Order ID for few orders in the Amazon FBA Returns Report due to attribution being triggered across different time zones. Initially, we were not including the offset in the return_date, which created challenges for the analytics team in accurately deduplicating these records. When partitioning by return_date in their deduplication queries, relying solely on _daton_batch_runtime was insufficient to resolve the issue, resulting in incorrect identification of duplicates.
• To address this issue, we have enhanced the data processing logic by incorporating the time zone offset into the return_date field. The offset is now captured and converted into its corresponding UTC value. For instance, a return date like "2024-07-03T09:13:33+06:00" is now stored as "2024-07-03T03:13:33" in UTC.
• This adjustment ensures that records with the same Order ID but different time zone offsets are accurately represented with distinct return_date values after conversion to UTC. This change significantly reduces confusion during the data deduplication process.
• By including the offset in the return_date field, the analytics team can now partition the data more effectively and eliminate duplicate records with greater accuracy. With the corrected return_date alongside _daton_batch_runtime, they can confidently perform deduplication.
• Example: For Order ID:12345, we had two records with different return_dates: one record with a return date of "2024-07-02T02:31:25-03:00" and another with a return date of "2024-07-01T22:31:25-07:00." After the changes, both return dates are converted to a single UTC representation, "2024-07-02T05:31:25." This allows the analytics team to partition the data effectively using return_date alongside _daton_batch_runtime, facilitating accurate deduplication.
• The changes have been implemented on the backend as of August 7th 2024, and return_date will now be consistently stored in UTC format. If you require historic data with return_date in UTC format, a rollback will be necessary.

Amazon Vendor Central

Table Name: RetailProcurementOrdersStatus

• The table was using changedAfter and changedBefore as date parameters to fetch data. However, Amazon does not support these date parameters for this table, causing it to behave as a full-load table without retrieving updated data.
• The date parameters have been switched to updatedAfter and updatedBefore to allow for precise retrieval of updated information. This modification ensures that all updated Purchase Order (PO) details are now being fetched correctly. For example, if an ASIN is updated after a PO is placed, the updated information will now be accurately captured.
• This change enables users to receive the most recent updates for Purchase Order details, improving the accuracy and relevance of the data retrieved.
• The changes have been implemented automatically as of August 9th, 2024, and the table will now function correctly with the new date parameters. If you require historic data, a rollback will be necessary.

Table Name: CatalogItems

• We have identified null values in the vendorDetails column of the Catalog Items table. This issue occurred because the vendorDetails attribute was excluded during a recent code revamp, resulting in it not being included in the API call.
• The vendorDetails attribute has been reintroduced, ensuring that data is now correctly populated in the vendorDetails column, and data for the vendorDetails column is now being received.
• Previously, the vendorDetails column received null values, while other columns continued to receive data. With the fix implemented, the vendorDetails column will now reflect accurate vendor information.
• Users will notice that the vendorDetails column now contains valid data, enhancing the overall reliability and completeness of the Catalog Items table.
• No action is required from users, as the changes have been automatically implemented and are functioning correctly. Please note that, since this is not a date-based table, we cannot retrieve past data. These changes were made on August 13th, 2024.

Unicommerce

• We released a new version of the Unicommerce connector. This update introduces several enhancements and new functionalities designed to significantly improve performance and data handling compared to the previous version. Key improvements include accelerated retrieval of historical data and expanded support for various reporting and data management needs.
• Added Support for 3 new reports:

Loris Data

• The CQ record field contains agent names and CQ scores, but each agent's name is being placed in a separate column. This is cluttering the data and unnecessarily increasing the number of columns and rows.

• We consolidated all agent names into a single column and their respective CQ scores into another column, ensuring the data populates correctly and more efficiently.
• This feature enables analysts to conduct data analysis more efficiently and present their findings to clients more effectively.
• Users will benefit from cleaner, more organized data in their tables, reducing the need for extensive SQL operations.
• Users can begin using this field, by reloading the table
• No changes are needed. The new column will be added and populated automatically.

Time Doctor

• Added support for activity_worklog table. This table will help in tracking the number of hours the users have entered in their work time. It gives a detailed worklog of a user.

New Connectors