Supplier price list import
Acme buys stock from a supplier called NordicGear, and NordicGear changes its prices from time to time. Each night the supplier posts a fresh price list, a simple spreadsheet-style file with one line per product, onto its own server. By the next morning Acme’s product catalogue should show those new prices. The supplier will not send the file to Acme, so Acme has to go and collect it: every night Acme logs in to the supplier’s server, downloads the file, checks it looks sensible, and updates the prices in its own database. The file is a full list each time, so collecting the same file twice does no harm. The case to guard against is a bad file, one that is cut short or garbled, because blindly loading it could wipe out good prices, so a file that fails its checks is set aside untouched and someone is told.
So the job runs itself once a night. Go to the supplier’s server, fetch the new price list, make sure it is complete and sane, and write the prices into Acme’s catalogue. Yesterday’s prices stay in force until a good file replaces them, and a file that fails its checks never touches the catalogue.
Here is how Art2link ESB builds that. This is the polling consumer shape with a file source. A Scheduler receive port (a receive port is the entry point that brings work onto the bus, the shared message backbone every flow publishes to and reads from) fires once a night. A send port (the exit point that hands a message to an outside system) bound to the SFTP Caller adapter (the connector that logs in to a file server) in two-way mode subscribes to the tick and issues a download command against the supplier’s server. A subscription is just the rule that decides which messages a port picks up. The fetched file rides back as the response and re-enters the bus through the port’s inbound flow, where a pipeline component (a small piece of code that runs on a message in flight) validates the CSV and disassembles it into the canonical PriceList JSON, classifying the emitted message as PriceList so the bus carries JSON, not CSV. A SQL Caller send port then writes the prices into the product database.
{{Config.PortName}} == "NightlyPriceClock"
A SQL send port subscribes to the canonical message and calls a stored procedure that upserts the whole file in one MERGE into the product table. Because the file is a full refresh, the import is naturally idempotent, fetching the same file twice converges on the same prices.
{{Message.MessageType}} == "PriceList"
When it fails. The two interesting failures are the fetch (server unreachable, file missing, host key changed) and the merge (malformed row, constraint violation). Set one shared Exception Message Type on both ports, PriceImportFailed, and subscribe an O365 Mail port to it for operations. A missing file at 02:00 is an email at 02:01, and yesterday's prices stay in force untouched; the held run waits in tracking for replay once the supplier re-publishes the file.
{{Message.MessageType}} == "PriceImportFailed"
Build it, step by step. The steps run in dependency order: every object is created before the object that selects it.
Under the application’s Message types, create the three types this flow routes on. A message type is a name plus a format, no schema required:
| Name | Format | Purpose |
|---|---|---|
| SupplierPriceList | CSV | the raw file as it arrives, the fallback type the disassembler component reads; never a map source |
| PriceList | JSON | the canonical price list the component emits and the rest of the estate consumes |
| PriceImportFailed | JSON | the exception type the failure path publishes under (Step 8) |
Three external parties, three credentials. Under the application’s Authentications, the dialog asks for a Name, the Adapter it pairs with, and a Definition, the credential shape that adapter offers, with the Application preset; the Definition decides the config section that follows.
| Setting | Value |
|---|---|
| Name | SupplierSftp |
| Adapter | SFTP Caller |
| Definition | SFTP Password Authentication |
| Username / Password | the supplier’s SFTP login |
| Host Key Fingerprint | the supplier server’s fingerprint |
| Setting | Value |
|---|---|
| Name | ProductDbSql |
| Adapter | SQL Caller |
| Definition | SQL Server Connection |
| Connection String (Database Config) | the product database’s connection string, credentials included |
| Setting | Value |
|---|---|
| Name | O365Ops |
| Adapter | O365 Mail Sender |
| Definition | Microsoft Graph |
| Tenant Id / Client Id / Client Secret (Graph AuthConfig) | an app registration with Mail.Send granted |
Create the table and procedure in the product database; the upsert port (Step 7) calls the procedure. Azure SQL’s OPENJSON reads the canonical PriceList body directly, so the whole file lands in one MERGE.
CREATE TABLE dbo.Products ( Sku VARCHAR(40) NOT NULL PRIMARY KEY, Price DECIMAL(18,2) NOT NULL, Currency CHAR(3) NOT NULL, UpdatedAt DATETIME2 NOT NULL CONSTRAINT DF_Prod_Upd DEFAULT SYSUTCDATETIME() ); CREATE OR ALTER PROCEDURE dbo.MergePriceList @PriceListJson NVARCHAR(MAX) AS BEGIN SET NOCOUNT ON; MERGE dbo.Products AS tgt USING ( SELECT sku, price, currency FROM OPENJSON(@PriceListJson, '$.items') WITH ( sku VARCHAR(40) '$.sku', price DECIMAL(18,2) '$.price', currency CHAR(3) '$.currency' ) ) AS src ON tgt.Sku = src.sku WHEN MATCHED THEN UPDATE SET Price = src.price, Currency = src.currency, UpdatedAt = SYSUTCDATETIME() WHEN NOT MATCHED THEN INSERT (Sku, Price, Currency) VALUES (src.sku, src.price, src.currency); END;
Under the application’s Pipeline components, create PriceListDisassembler. It does two jobs in one pass on the raw file: it validates (header shape, a minimum row count, a positive-price check) and then disassembles the CSV into the canonical PriceList JSON, classifying the emitted message as PriceList so the bus carries JSON, not CSV. This is required: a CSV body can never be a map source (see the note above), so the parse has to happen in a component before anything else runs. A file that fails validation returns Success = false, which suspends the run as a poison message before it can touch the catalogue. Both validation settings default empty, so a missed setting fails loudly instead of waving a bad file through.
using System.ComponentModel.DataAnnotations; using System.Text.Json; using CC.Art2link.Pipelines.Domain.Models.PipelineComponents; public sealed class PriceListDisassemblerConfig { [Range(1, int.MaxValue)] public int MinRows { get; set; } [Required] public string ExpectedHeader { get; set; } = string.Empty; [Required] public string Supplier { get; set; } = string.Empty; } public sealed class PriceListDisassembler : PipelineComponentBase<PriceListDisassemblerConfig> { public override string Name => "PriceListDisassembler"; protected override Task<PipelineComponentOutput> ExecuteAsync( PipelineComponentInput input, PriceListDisassemblerConfig config, CancellationToken cancellationToken) { // 1. Read the raw CSV body the SFTP download handed in. var lines = input.Body .Split('\n') .Select(l => l.TrimEnd('\r')) .Where(l => l.Length > 0) .ToList(); // 2. Validate before parsing. if (lines.Count == 0 || lines[0] != config.ExpectedHeader) return Fail("price list header missing or unexpected"); if (lines.Count - 1 < config.MinRows) return Fail($"price list has {lines.Count - 1} rows, " + $"expected at least {config.MinRows}"); // 3. Disassemble the CSV rows into the canonical PriceList shape. var items = new List<object>(); string? validFrom = null; foreach (var row in lines.Skip(1)) { var cols = row.Split(','); if (cols.Length < 5 || !decimal.TryParse(cols[2], out var price) || price <= 0m) return Fail($"invalid price in row: {row}"); validFrom ??= cols[4]; items.Add(new { sku = cols[0], price, currency = cols[3] }); } var json = JsonSerializer.Serialize(new { supplier = config.Supplier, validFrom, items }); // 4. Emit one JSON message and classify it as the canonical type, // so the bus carries PriceList JSON, never the raw CSV. return Task.FromResult(new PipelineComponentOutput { Success = true, Messages = [new PipelineMessage { Body = json, MessageType = "PriceList" }] }); static Task<PipelineComponentOutput> Fail(string msg) => Task.FromResult(new PipelineComponentOutput { Success = false, ErrorMessage = msg }); } }
The supplier’s prices.csv as it arrives:
sku,description,unit_price,currency,valid_from TP-4501,Trail Pack 45L,61.20,EUR,2026-06-05 HX-0090,Hex Stove,18.75,EUR,2026-06-05 GL-1240,Glacier Lantern,9.40,EUR,2026-06-05
The canonical PriceList JSON the component emits and classifies as PriceList:
{
"supplier": "NordicGear",
"validFrom": "2026-06-05",
"items": [
{ "sku": "TP-4501", "price": 61.20, "currency": "EUR" },
{ "sku": "HX-0090", "price": 18.75, "currency": "EUR" },
{ "sku": "GL-1240", "price": 9.40, "currency": "EUR" }
]
}Create a receive port NightlyPriceClock on the Scheduler; its tick is what wakes the fetch port (Step 6).
| Setting | Value |
|---|---|
| Adapter | Scheduler |
| Repeat Every | 1 |
| Repeat Unit | Days |
| Daily Window | opening 02:00:00 AM |
Create a send port PriceListFetch on the SFTP Caller, two-way mode, with the download command for /outbound/prices.csv. Subscription: {{Config.PortName}} == "NightlyPriceClock". (The dropdowns can also create types and components inline; building the dependencies first keeps each dialog a selection.)
| Setting | Value |
|---|---|
| Adapter | SFTP Caller |
| Way | Two |
| Host | the supplier’s server |
| Command | Download |
Set Authentication to SupplierSftp, from Step 2.
The fetched file rides back as the response and re-enters the bus through the port’s Response stages. Set Message Type Fallback to SupplierPriceList, from Step 1, so the raw CSV body carries a type, then reference the PriceListDisassembler, from Step 4, in the inbound flow. On the component reference set ExpectedHeader to sku,description,unit_price,currency,valid_from, MinRows to a floor that would catch a truncated file, e.g. 10, and Supplier to NordicGear. There is no map here: the component validates the CSV and emits the canonical PriceList JSON, classifying its output as PriceList, so what lands on the bus is JSON.
Create a send port PriceUpsert on the SQL Caller, one-way, calling the merge procedure from Step 3. Subscription: {{Message.MessageType}} == "PriceList".
| Setting | Value |
|---|---|
| Adapter | SQL Caller |
| Way | One |
| Command Type | StoredProcedure |
| Command Text | dbo.MergePriceList |
Set Authentication to ProductDbSql, from Step 2.
Its single Input Parameters row:
| Parameter | Value |
|---|---|
| PriceListJson | {{Message.Body}} |
On both send ports, PriceListFetch and PriceUpsert, set Exception Message Type to PriceImportFailed, from Step 1.
Create an O365 Mail port OpsAlert subscribing on {{Message.MessageType}} == "PriceImportFailed":
| Setting | Value |
|---|---|
| Authentication | O365Ops, from Step 2 |
Everything you configure is live the moment you save it, there is nothing to deploy. Start the send ports first, PriceUpsert, OpsAlert and PriceListFetch, then the receive port NightlyPriceClock.
Set Tracking to Enabled + Body on every port the flow touches, so you can walk each run step by step, with its message body, in tracking.
Drop a sample prices.csv on the supplier server (or your test SFTP), and trigger a tick. To avoid waiting for the 02:00 window, temporarily set NightlyPriceClock to Repeat Every 1 / Minutes, then restore the nightly window once the test passes. Verify in tracking: fetch → disassemble to PriceList JSON → merge, and the product table updated.
Re-run the same file and confirm prices converge unchanged.
Remove the file and confirm the 02:01 alert email.