Configuring Imports

An import is the data destination in a Celigo integration. It takes records from an upstream step and writes them to an external system -- REST APIs, databases, ERPs, file servers, or AI models. Every import is bound to exactly one connection and one adaptor type.

Imports handle six concerns:

Field mapping -- transforming source fields into the destination system's expected format (including value resolution via static maps and lookup tables). Uses Mapper 2.0 (
```
mappings[]
```
array) by default; NetSuite and Salesforce imports only support Mapper 1.0 (
```
mapping.fields[]
```
/
```
mapping.lists[]
```
)
Operation logic -- create, update, upsert, delete, attach/detach
Hooks -- JavaScript pre/post processing at various pipeline stages (preMap, postMap, postSubmit). File-based imports that generate files from records also support postAggregate
One-to-many -- fan out child records from a parent. Set
```
oneToMany: true
```
and
```
pathToMany
```
to the child array path (e.g.,
```
"lineItems"
```
) when one source record should create multiple import operations
Response mapping -- extract fields from the import's API response back into the record for downstream steps. Configured on the flow's
```
pageProcessors[]
```
entry, but planned when building the import. The response is available via
```
_json
```
(the raw API response) and
```
errors
```
. Use
```
_json.fieldName
```
to extract from the response (e.g.,
```
_json.id
```
for a created record's ID,
```
_json.output.1.content.0.text
```
for OpenAI responses). Response mapping uses Transformation 1.0 syntax (extract/generate pairs), not the newer expression-based transforms
postResponseMap hook -- JavaScript processing after response mapping merges the response back into the record. Configured on the flow's
```
pageProcessors[]
```
entry, but planned when building the import. Use to transform or enrich the merged record before downstream steps

Imports are used across flows, APIs, and tools.

Import Execution Pipeline

When records arrive at an import step, this pipeline executes in strict order:

Input filter (optional) -- discards records before any processing (configured as an expression on the flow's
```
pageProcessors[]
```
entry)
preMap hook (optional) -- JavaScript processing before field mapping
Field mapping -- Mapper 2.0 or 1.0 maps source fields to destination fields, including lookups and hardcoded values
postMap hook (optional) -- JavaScript processing after field mapping, before submission
Submit to destination -- writes the mapped record to the external system
postSubmit hook (optional) -- JavaScript processing after the destination responds (access response data, log results, trigger side effects)
Response mapping (optional) -- carries data from the destination response back into the record for downstream steps. Configured on the flow's
```
pageProcessors[]
```
entry, not on the import itself
postResponseMap hook (optional) -- JavaScript processing after response mapping merges response data back into the record

Key distinction: Response mapping lives on the flow's

pageProcessors[]

entry, not on the import resource. When building an import that needs to pass data downstream, plan the response mapping at flow design time.

Categories of Import

Record-Based Imports

Submit structured records to APIs, databases, or ERPs. The vast majority of imports.

```
NetSuiteDistributedImport
```
-- high-performance SuiteApp writes (add, update, addupdate, delete, attach, detach)
```
HTTPImport
```
-- REST/GraphQL APIs (POST, PUT, PATCH, DELETE). Supports connector-assisted (
```
formType: "assistant"
```
) and GraphQL (
```
graph_ql
```
) modes
```
SalesforceImport
```
-- Salesforce CRUD via SOAP, REST, Bulk, or Composite Record API
```
RDBMSImport
```
-- SQL databases (Snowflake, PostgreSQL, MySQL, SQL Server, Oracle). Uses
```
per_record
```
,
```
bulk_insert
```
, or
```
bulk_load
```
query types
```
MongodbImport
```
,
```
DynamodbImport
```
,
```
JDBCImport
```
-- other databases

File-Based Imports

Write or upload files to remote storage. Require the

file{}

configuration block. Two modes:

Record-to-file -- aggregates incoming records into a file (CSV, JSON, XML, XLSX). The
```
file{}
```
block defines the output format.
Blob passthrough -- transfers a binary blob as-is from an upstream export. Set the
```
blobKeyPath
```
field to the path in the record that contains the blob key.

Adaptor types:

```
HTTPImport
```
with
```
http.type: "file"
```
-- upload files over HTTP to cloud storage APIs (Google Drive, Box, Dropbox, Azure Blob Storage)
```
FTPImport
```
-- CSV, XML, JSON, XLSX, EDI files to FTP/SFTP
```
S3Import
```
-- objects to Amazon S3
```
AS2Import
```
-- AS2 EDI file transmission
```
FileSystemImport
```
-- local/on-premise filesystem writes

AI Imports

Invoke AI models for classification, extraction, or safety checks. No

_connectionId

required unless using BYOK.

```
AiAgentImport
```
-- OpenAI or Gemini model invocations with structured output, tool use, and reasoning
```
GuardrailImport
```
-- PII detection, content moderation, or custom AI-based validation

Stack and Tool Imports

```
WrapperImport
```
-- custom pre-built stack connectors (Walmart, BigCommerce)
```
ToolImport
```
-- invoke a Celigo Tool resource

Import Matching (create / update / upsert)

The core decision on most record-based imports is the operation -- what happens to each record at the destination. Create requires no match key; update and delete require one; upsert checks first and does whichever applies. Users describe the intent in business terms ("look up the customer and update them", "match by email and upsert", "skip the ones that already exist") that resolve into five behaviors:

Always create -- every record is submitted as new. No matching, no checks.
Create only if missing -- match-check first; submit as create if not found, skip silently if found.
Update only if exists -- match-check first; submit as update if found, skip silently if not.
Update only if exists, fail if missing -- strict variant; no-match records error out instead of skipping.
Upsert -- submit a create when no match is found, an update when matched. The catch-all, and the right default when the user is vague.

When matching applies, decide four things: the matching behavior, the match key field(s) (

email

external_id

customer.id

-- required for anything other than always-create), the on-match action (update, skip, fail), and the on-no-match action (create, skip, fail).

How a destination implements matching is adaptor-specific -- there is no single "matching mode" field. A destination might expose: a native upsert keyed off an external ID (Salesforce upsert, NetSuite upsert, RDBMS

ON CONFLICT

); a distinct

addupdate

operation that handles both paths in one call; an

ignoreExisting

flag paired with a lookup that probes before writing; two separate create and update endpoints with no upsert variant (see composite imports below); or a lookup endpoint plus separate create and update endpoints, where the lookup runs pre-write to drive the create-vs-update decision. Some destinations have no matching concept at all -- writing a CSV to FTP, sending an email, posting to a webhook -- so every record goes out as-is.

Prefer import-level matching over a separate lookup step. Imports natively support this pre-write check, so a single import that does the matching and the write together means fewer steps, fewer round-trips, and no glue logic to maintain. A standalone lookup earns its place only when the looked-up data has a consumer beyond the write -- a router branching on something other than "does this exist", an AI agent reasoning over the result, or multiple downstream steps reading different fields. If the only consumer is the destination call itself, the work belongs inside the import.

Composite (two-endpoint) imports

When an HTTP destination has no native upsert but exposes separate create and update endpoints, a composite import pins both endpoints on a single import node, role-tagged create and update. The runtime picks per record via a match-key check -- the same way a native upsert would -- so the flow stays one step. Prefer this over two separate imports driven by an upstream lookup or router; reach for separate imports only when the create and update paths must diverge beyond endpoint selection (different mappings, different downstream consumers, or different hook chains).

Quick Reference

Adaptor Decision Matrix

Your data goes to...	Use adaptorType	Category	Read schema
REST or GraphQL API	`HTTPImport`	Record-based	http.yml
NetSuite (any method)	`NetSuiteDistributedImport`	Record-based	netsuitedistributed.yml
Salesforce objects	`SalesforceImport`	Record-based	salesforce.yml
SQL database (Snowflake, PostgreSQL, etc.)	`RDBMSImport`	Record-based	rdbms.yml
MongoDB	`MongodbImport`	Record-based	mongodb.yml
DynamoDB	`DynamodbImport`	Record-based	dynamodb.yml
JDBC database (non-built-in)	`JDBCImport`	Record-based	jdbc.yml
Files over HTTP (Google Drive, Box, Dropbox, Azure Blob)	`HTTPImport` with `http.type: "file"`	File-based	http.yml
Files to FTP/SFTP	`FTPImport`	File-based	ftp.yml
Files to S3	`S3Import`	File-based	s3.yml
AS2 EDI transmission	`AS2Import`	File-based	as2.yml
Local filesystem	`FileSystemImport`	File-based	filesystem.yml
OpenAI / Gemini	`AiAgentImport`	AI	aiagent.yml
PII detection / content moderation	`GuardrailImport`	AI	guardrail.yml
Celigo Tool	`ToolImport`	Tool	wrapper.yml
Pre-built stack connector	`WrapperImport`	Stack	wrapper.yml

Raw HTTP is the fallback, not the default. Pick the most specific match, in order:

Native adaptor -- if the application has its own row (NetSuite, Salesforce, databases, FTP/S3), use it. Do not build an
```
HTTPImport
```
against that app's REST API.
Pre-built HTTP connector -- for any other REST/GraphQL app, check the 550+ connector catalog before writing HTTP config (see Check for a pre-built connector). The step is still an
```
HTTPImport
```
, but it runs on a connector-backed connection and takes its endpoint config from the connector.
Manual HTTP -- hand-write the config from public API docs only when no connector exists or it doesn't cover the operation you need.

adaptorType

is case-sensitive:

NetSuiteDistributedImport

, not

netsuitedistributedimport

Minimum Required Fields

Every import needs at minimum:

name

adaptorType

_connectionId

(except AiAgentImport/GuardrailImport without BYOK), and the adaptor config block (

http{}

netsuite_da{}

salesforce{}

, etc.).

Which Schemas to Read

Always: request.yml (base fields)
Plus: the adaptor-specific file from the decision matrix
If file-based: also file.yml
If cloning: clone-request.yml, clone-response.yml

Schema Index

All schemas are in references/schemas/:

Base fields (all imports): request.yml
Response shape: response.yml
Adaptor-specific config:
- http.yml -- HTTP/REST/GraphQL (methods, URIs, headers, response parsing, upsert via existingExtract)
- netsuitedistributed.yml -- NetSuite SuiteApp (operation, recordType, internalIdLookup, mapping, lookups)
- netsuite.yml -- NetSuite legacy
- salesforce.yml -- Salesforce (sObjectType, operation, api, idLookup)
- rdbms.yml -- SQL databases (queryType, query, bulkInsert, bulkLoad)
- ftp.yml -- FTP/SFTP
- s3.yml -- Amazon S3
- mongodb.yml -- MongoDB (method, collection, filter, upsert)
- dynamodb.yml -- DynamoDB
- jdbc.yml -- JDBC databases
- as2.yml -- AS2 EDI
- wrapper.yml -- custom stack connectors
- filesystem.yml -- local filesystem
AI config:
- aiagent.yml -- AI agent (provider, model, instructions, tools, structured output)
- guardrail.yml -- guardrails (PII, moderation, AI-agent validation)
File output: file.yml (CSV, XML, JSON, XLSX config for file-based imports)
Clone: clone-request.yml, clone-response.yml

Related Skills

configuring-connections > Quick Reference -- connection types and auth for import destinations
writing-mappings > Mapper 2.0 Workflow -- field mappings on imports
writing-scripts > Data Pipeline Hooks -- preMap, postMap, postSubmit, postAggregate hooks
writing-handlebars > Quick Reference -- dynamic values in URIs, HTTP bodies, SQL queries
building-flows > How to Build a Flow -- wiring imports into flow pipelines as page processors
troubleshooting-flows > Diagnostic Workflow -- diagnosing import-related failures

How to Build an Import

1. Identify the target application

What system are you writing data to? This determines adaptor type, connection type, and configuration shape.

2. Check for existing patterns

Before building from scratch, look at what already exists:

bash

# Search your account (fast, uses local index)
celigo account search "<keyword>"

# Show what an existing import uses (connection) and what uses it (flows)
celigo account dependencies import <id>

# Find orphaned imports not referenced by any flow
celigo account lint

# Search marketplace templates
celigo templates marketplace

# Extract just imports from a template
celigo templates preview <id> --model Import
celigo templates preview <id> --summary

The account index auto-refreshes when stale (>4 hours). Force a fresh snapshot with

celigo account snapshot

3. Check for a pre-built connector

Always run this check before writing any HTTP config. Celigo maintains 550+ HTTP connectors with pre-configured auth, endpoints, and resources. Hand-write a manual

HTTPImport

from public API docs only when this search comes up empty or the connector doesn't cover the operation you need.

bash

# Search HTTP connectors
celigo http-connectors list | grep -i "<application-name>"
celigo http-connectors get <id> --full    # see endpoints, resources, auth config

# Drill into the endpoints the connector defines for imports
celigo http-connectors catalog <id> --resource-type import --published-only
celigo http-connectors endpoint-detail <id> --resource-type import --resource-id <rid> --endpoint-id <epid>

# Search trading partner connectors (EDI, AS2)
celigo tp-connectors list

If a connector exists, create the connection from it (

http._httpConnectorId

-- see configuring-connections > Check for a pre-built connector and global iClient) and take the import's

relativeURI

, method, and body/response shapes from the connector's endpoint metadata rather than reconstructing them from public API docs. The connector-reference fields on the import itself (

http._httpConnectorEndpointId

http._httpConnectorVersionId

http._httpConnectorResourceId

) are read-only -- the platform sets them; what you control is the connection and the endpoint config you copy from the connector.

4. Query metadata for the target system

For NetSuite, Salesforce, and RDBMS connections, discover available record types and fields:

bash

celigo metadata types <connectionId>          # List record types / sObjects / tables
celigo metadata fields <connectionId> <type>  # List fields for an entity type

NetSuite:
```
metadata fields
```
returns field IDs, types, and groups — use the IDs for
```
mapping.fields[].generate
```
and
```
mapping.lists[].fields[].generate
```
. Sublist names (e.g.,
```
"item"
```
,
```
"addressbook"
```
) appear as groups, which map to
```
mapping.lists[].generate
```
. Lookup field IDs here are the
```
searchField
```
/
```
resultField
```
values for
```
netsuite_da.lookups[]
```
.
Salesforce:
```
metadata fields
```
returns field API names, types, and relationship info. Use field API names for
```
salesforce.sObjectType
```
lookups and for discovering which fields are createable/updateable.
RDBMS:
```
metadata fields
```
returns column names and types for a table — use these to write SQL queries (see
```
writing-sql
```
) and verify column names before building
```
bulkInsert.tableName
```
or
```
bulkLoad.tableName
```
.

5. Determine the category

Is this a record-based import (submit records to an API/database/ERP), a file-based import (write files to storage), or an AI import (invoke a model)?

6. Choose the right adaptor type

Refer to the Adaptor Decision Matrix in the Quick Reference above.

7. Build the import JSON

Reference the Schema Index for the exact fields needed. Use the Which Schemas to Read decision rule to determine which files to consult.

File Uploads over HTTP (multipart/form-data)

Some destination APIs accept files only as

multipart/form-data

POSTs (Jira attachments, QuickBooks attachables, OpenAI file uploads). This is an

HTTPImport

in file-transfer mode (

http.type: "file"

) and works nothing like a JSON record write. Four pieces have to line up:

Where the bytes come from. The import never carries the file itself. A preceding blob export or blob lookup pulls the file into blob storage, and that step's response mapping puts the reference onto the record -- the idiom is
```
{"extract": "data.0.blobKey", "generate": "blobKey"}
```
. The record then carries a
```
blobKey
```
(a storage pointer, not content).
The media type. The connection's media type (or the import's request media type override) is
```
multipart/form-data
```
; success/error response media types usually override to JSON so replies parse normally.
The request body. Not raw MIME and not a normal handlebars payload -- a JSON array of parts, each
```
{name, value, type}
```
plus optional
```
filename
```
(include the extension) and
```
mime-headers
```
. The file part is
```
"type": "attachment"
```
with
```
"value": "{{blob}}"
```
--
```
{{blob}}
```
is the only accepted value for an attachment (anything else 422s). Fields the API wants alongside the file ride as
```
"type": "inline"
```
parts; an inline part whose value is a JSON object must be serialized. One file reference per import.
blobKeyPath
(Advanced settings) -- the JSON path in the record where the blobKey lives (
```
blobKey
```
, or
```
file.blobKey
```
if nested). At send time the platform follows it into blob storage and streams the real bytes into the attachment part.

The platform assembles the final MIME body itself -- it generates the

boundary

(never hardcode one), writes each part's

Content-Disposition

, and substitutes the attachment part with the raw file bytes. The parts array is a build recipe, not the payload.

Not every multipart API is form-data. Some upload endpoints expect

multipart/related

instead (e.g. Google Drive's

/upload/drive/v3/files?uploadType=multipart

), and the parts-array machinery does NOT apply. There the request body is the literal MIME document: explicit boundary, a JSON metadata part, and a content part referencing

{{blob}}

(double braces). Check which flavor the destination API documents before building -- mixing them produces an import that saves cleanly and fails at runtime.

Async Destinations (submit, poll, confirm)

Some destination APIs only acknowledge a write (an HTTP 202, a job ticket) and finish it in the background -- bulk loads, file ingestion, document conversion. Attach an async helper to the import (

http._asyncHelperId

) so the step submits, polls a status export until the external work completes, and only then resolves. The mechanics and constraints (a status export with done/error value lists and poll intervals; no transform, output filter, or hook on the async-configured step; helpers cannot nest) are identical to the export side -- see configuring-exports > Async APIs (submit, poll, fetch). Only add one when the API genuinely cannot confirm the write synchronously.

CLI Commands

bash

# CRUD
celigo imports list
celigo imports get <id>
celigo imports create < import.json
celigo imports update <id> < import.json
celigo imports set <id> key=value [key2=value2 ...]
celigo imports delete <id> [-y]

# Invoke (test submission without creating a job)
echo '[{"name":"test"}]' | celigo imports invoke <id>

# Clone and connection management
echo '{"connectionMap":{"oldConnId":"newConnId"}}' | celigo imports clone <id>
celigo imports replace-connection <id> <newConnectionId>

# Discovery
celigo templates marketplace
celigo http-connectors list
celigo http-connectors catalog <id> --resource-type import --published-only
celigo http-connectors endpoint-detail <id> --resource-type import --resource-id <rid> --endpoint-id <epid>
celigo tp-connectors list
celigo metadata types <connectionId>
celigo metadata fields <connectionId> <entityType>

# Debug
celigo imports enable-debug <id> [--duration <minutes>]
celigo imports disable-debug <id>

Pre-Submit Checklist

Required (all imports)

```
name
```
is set
```
adaptorType
```
exact case matches connection type (request.yml > adaptorType)
```
_connectionId
```
references a valid, online connection (skip for AI imports without BYOK)
Adaptor config block name matches adaptorType (
```
http{}
```
for HTTPImport,
```
netsuite_da{}
```
for NetSuiteDistributedImport, etc.)

Adaptor-specific

HTTP: pre-built connector was checked (
```
celigo http-connectors list
```
) -- hand-written config only because no connector covers the app or operation
HTTP:
```
http.method
```
and
```
http.relativeURI
```
are set (http.yml)
NetSuite:
```
netsuite_da.operation
```
and
```
netsuite_da.recordType
```
are set (netsuitedistributed.yml)

RDBMS:

rdbms.queryType

per_record

bulk_insert

-- NOT legacy

insert

update

(rdbms.yml)

Salesforce:

salesforce.sObjectType

and

salesforce.operation

are set (salesforce.yml)

Cross-resource consistency

Connection
```
type
```
matches the import's
```
adaptorType
```
If response mapping needed: configured on the flow's
```
pageProcessors[]
```
entry, not on the import itself
If one-to-many:
```
oneToMany: true
```
and
```
pathToMany
```
is set to the child array path
If using Mapper 1.0 (NetSuite/Salesforce):
```
mapping.fields[]
```
/
```
mapping.lists[]
```
, not
```
mappings[]
```

Gotchas

PUT erases omitted fields. Always GET first, modify, then PUT. The
```
set
```
command handles this.
Including a
rest:
block creates a legacy RESTImport. Use only
```
http:
```
for new imports.
Input filter skips that import, not the record. Filtered records skip the current import step but continue to subsequent steps in the flow. They aren't dropped -- check
```
numIgnore
```
on the job if records seem to bypass a step.
Multipart file parts only accept
{{blob}}
. In a
```
multipart/form-data
```
parts array, the file part must be
```
"type": "attachment"
```
with
```
"value": "{{blob}}"
```
-- any other value 422s. Never hardcode the MIME
```
boundary
```
; the platform generates it. Fields the API wants alongside the file ride as
```
"type": "inline"
```
parts.
bodyKey
/
blobKey
in logs is an artifact, not a payload field. Audit and debug logs never show the assembled multipart body -- an internal storage pointer appears where the body would be. But if the destination actually received the literal string
```
bodyKey
```
or
```
blobKey
```
, the file part is misconfigured (an
```
inline
```
part where an
```
attachment
```
belongs, or a
```
blobKeyPath
```
that doesn't resolve).

Common Errors

Error	Cause	Fix
422 `adaptorType invalid`	Wrong case	Use exact case from decision matrix: `HTTPImport` , `NetSuiteDistributedImport` , etc.
422 `_connectionId required`	Missing connection	Set `_connectionId` to a valid connection ID
422 `queryType invalid`	Legacy Snowflake value	Use `per_record` or `bulk_insert` , not `insert` / `update`
422 `distributed required`	Missing NetSuite flag	Use `NetSuiteDistributedImport` with `distributed: true` on connection
422 `mapping invalid`	Wrong mapper version	NetSuite/Salesforce use Mapper 1.0 ( `mapping.fields[]` ), not Mapper 2.0 ( `mappings[]` )
422 attachment `value` invalid	Multipart file part is not `{{blob}}`	Set the file part to `"type": "attachment"` , `"value": "{{blob}}"` ; let the platform generate the `boundary`

configuring-imports

NPX Install

Tags

SKILL.md Content

Configuring Imports

Import Execution Pipeline

Categories of Import

Record-Based Imports

File-Based Imports

AI Imports

Stack and Tool Imports

Import Matching (create / update / upsert)

Composite (two-endpoint) imports

Quick Reference

Adaptor Decision Matrix

Minimum Required Fields

Which Schemas to Read

Schema Index

Related Skills

How to Build an Import

1. Identify the target application

2. Check for existing patterns

3. Check for a pre-built connector

4. Query metadata for the target system

5. Determine the category

6. Choose the right adaptor type

7. Build the import JSON

File Uploads over HTTP (multipart/form-data)

Async Destinations (submit, poll, confirm)

CLI Commands

Pre-Submit Checklist

Required (all imports)

Adaptor-specific

Cross-resource consistency

Gotchas

Common Errors