Hi All,
I have a scenario where we load data from a third-party system through an API integration on a daily basis In ODC application.
However, there are cases where the third-party API token expires, which causes the daily data load to fail and results in missing data in our application.
In such cases, what would be the best approach to load the missing data from previous dates once the token is renewed?
Do we need to maintain a separate database/table to track the sync status for each day, or is there another recommended approach for handling this?
I also want to ensure that the AO's are saved properly and can be recovered/reprocessed if a daily sync fails.
Please suggest the best approach for designing this process.
Hi @Vinod Kumar ,
If you don't want to use an Entity for the daily sync, you can use an alternative approach with Settings.
Create the following two settings:
In your logic:
With this approach, you don't need to create an additional Entity or add extra database logic for the sync.
You can simply update the settings from the ODC Portal whenever you want to perform a manual data load and then trigger the sync.Hope this will help you.
Though this works, it needs manual intervention every time something goes wrong. That's not what you'd want with expiring tokens.
Hi Vinod,
Agree with Kilian, I'd avoid manual steps here. What usually works well:
1) Create a small SyncLog entity (SyncDate, Status, Attempts, LastError).
2) The timer doesn't load only "today": it loads every date from the last successful SyncDate up to today, so any failed day is picked up automatically on the next run.
3) Make the load idempotent (upsert by the external record ID), so reprocessing a day never duplicates data.
4) Handle the token inside the flow: get a fresh token (or use the refresh token) before calling the API and retry once on 401. That way an expired token doesn't break the sync at all.
If a day keeps failing, the log shows it and you can alert someone instead of silently losing data.