Product Master Data Sync from E-Commerce Middle Platform to Finance ERP: A Practical Guide on a Single Incremental Strategy with Qeasy
What This Strategy Solves
Pushing product master data from an e-commerce middle platform to a cloud ERP looks like a trivial "create material" job, but in real projects we routinely see the two sides fall out of sync within three months. For a retail enterprise where the e-commerce middle platform is the single source of truth for products, the downstream finance ERP relies on it for material records, units, and categories. This means the sync strategy must be stable, re-runnable, and easy to troubleshoot. We use the Qeasy data integration platform to implement it as an independent sync strategy so it does not interfere with other master data and is easy to maintain on its own.
Data Flow and Field Mapping
The overall flow is "E-commerce Middle Platform → Qeasy Middle Layer → Cloud ERP". The middle platform provides key fields such as product code, name, specification, unit, category, and modification time; Qeasy handles extraction, cleansing, and mapping; the cloud ERP receives the payload and persists it as material records.
Key field mapping (only the business-critical fields are shown):
| Business Meaning | Source (Middle Platform) | Target (Cloud ERP) | Mapping Notes |
|---|---|---|---|
| Product code | number | number | Primary key, must stay consistent |
| Name | name | name | Direct mapping |
| Specification | spec | spec | Pass-through as text |
| Base unit | unit | baseunit | Code mapped to unit ID |
| Category | category | category | Code mapping |
| Modification time | modify_time | — | Used as incremental cursor only |
| Internal ID | id | — | Used for idempotency |
How to Configure on Qeasy
The source endpoint is a GET product list API with page / page_size pagination (default 20). Incremental extraction is driven by a modification time range; the parameters modify_start_time and modify_end_time are automatically substituted by Qeasy with {{LAST_SYNC_TIME}} and {{CURRENT_TIME}}. A detail endpoint is associated as detailAPI to enrich extended fields on demand.
The target endpoint is configured as the cloud ERP material write API (POST / EXECUTE), with the request body assembled according to the target schema. On Qeasy we centralize code mapping: category and unit mapping tables live in the platform's "Mapping Table" module rather than scattered in scripts, so that the customer's business users can adjust them later without developer involvement.
Implementation Steps
Phase 1: Incremental Start Point. For the first run, pick a clear historical timestamp as LAST_SYNC_TIME and do a full backfill. After that, each scheduled run uses the last successful time as the start and CURRENT_TIME as the end. The source crontab is set to every 3 hours (4 */3 * * *); running frequently keeps the incremental window short and makes recovery easy when something goes wrong.
Phase 2: Full Trigger. In addition to scheduled incremental runs, Qeasy allows manual triggering of a full backfill for initialization or drift repair. We recommend keeping full and incremental as two separate entry points with independent scheduling.
Phase 3: Schedule Frequency. Product master data does not change very often, so every 3 hours is a safe default; if the volume is large and changes are frequent, drop it to every hour. A common pattern we see in Qeasy projects is "incremental and full in parallel": incremental keeps data fresh, full acts as a reconciliation backstop.
Pitfalls We Have Hit
Pitfall 1: Cursor timestamp precision. The source API returns timestamps in milliseconds, but the input template requires three extra zeros to convert to a microsecond string. If you forget this, the source system compares strings lexicographically and the time window goes wrong.
Pitfall 2: Pagination misses data. When products are sorted by modification time and paged, multiple updates within the same second can be duplicated or skipped. The safe approach is to use the product code as the idempotency key on Qeasy and ignore re-entries.
Pitfall 3: Unit and category code mismatch. The middle platform sends display names; the cloud ERP expects codes. We have seen customers hard-code the mapping inside scripts and then have to touch the code every time a unit changes, which is why we strongly recommend centralizing mapping in Qeasy's mapping tables.
Pitfall 4: Detail API silently returns empty. The detail API occasionally returns an empty body with HTTP 200. With autoFillResponse=true in the source config, Qeasy treats it as success and skips it, leaving extended fields blank. Add an "empty response detection" check on Qeasy to catch this.
Pitfall 5: Schedule time zone. The crontab expression is interpreted in the server's time zone. For cross-time-zone projects, always confirm the time zone setting on the strategy in Qeasy, otherwise the early-morning window drifts into business hours.
Suitable and Unsuitable Scenarios
Suitable: The e-commerce middle platform is the single source of truth for products and the downstream finance ERP needs stable material records; product changes are infrequent but must be available the next day; you want a single, re-runnable, traceable strategy to maintain.
Not suitable: Products affect orders and inventory in real time (use an instant inventory or order strategy instead); the catalog is very large and changes constantly (switch to change-log subscription); the upstream and downstream coding systems are fundamentally incompatible (fix master data governance first, then talk about sync).