Emmanuel Corels
← Blog
Product Engineering

POST JSON with curl—and make failures visible to automation

Generate valid JSON without shell-quoting surprises, send the correct media type, preserve error bodies, capture status and retry only when duplication is controlled.

By Emmanuel Corels

A curl command prints a helpful JSON validation error yet a deployment script exits successfully. Another command sends JSON-looking text with application/x-www-form-urlencoded. A third leaks a bearer token through the process list. A production-grade API probe must get four things right: bytes, media type, authentication handling and failure semantics.

Use curl’s JSON shortcut when available

curl --silent --show-error --fail-with-body   --json '{
    "customer_id": "cus_123",
    "items": [{"sku": "router-8", "quantity": 2}]
  }'   https://api.example.test/v1/orders

curl 7.82 introduced --json. It is shorthand for binary data plus Content-Type: application/json and Accept: application/json. curl does not validate or reformat the JSON; malformed input remains malformed.

Prefer a reviewed payload file over fragile shell quoting

jq -n   --arg customer_id "$CUSTOMER_ID"   --arg sku "$SKU"   --argjson quantity "$QUANTITY"   '{
    customer_id: $customer_id,
    items: [{sku: $sku, quantity: $quantity}]
  }' > payload.json

jq -e . payload.json >/dev/null

curl --silent --show-error --fail-with-body   --json @payload.json   https://api.example.test/v1/orders

Do not interpolate untrusted strings into JSON syntax. A JSON-aware generator handles quotes, backslashes and Unicode correctly. Protect payload files if they contain personal or confidential data, and remove them according to the job’s retention policy.

Understand the older portable form

curl --silent --show-error --fail-with-body   --header 'Content-Type: application/json'   --header 'Accept: application/json'   --data-binary @payload.json   https://api.example.test/v1/orders

--data implies POST but defaults to application/x-www-form-urlencoded. --data-binary preserves file bytes, including newlines. The explicit headers communicate the representation expected by the API.

Make HTTP errors change the process exit status

By default, curl can exit zero after receiving HTTP 404 or 500 because the transfer itself succeeded. --fail-with-body returns curl error 22 for HTTP 400 or greater while retaining the response body for diagnosis.

response_file=$(mktemp)
headers_file=$(mktemp)
trap 'rm -f -- "$response_file" "$headers_file"' EXIT

set +e
http_code=$(
  curl --silent --show-error --fail-with-body     --output "$response_file"     --dump-header "$headers_file"     --write-out '%{http_code}'     --json @payload.json     https://api.example.test/v1/orders
)
rc=$?
set -e

if (( rc != 0 )); then
  printf 'request failed: curl=%d http=%s
' "$rc" "$http_code" >&2
  jq . "$response_file" 2>/dev/null || cat "$response_file" >&2
  exit "$rc"
fi

Capture the exit status immediately. The example temporarily disables set -e so the script can retain curl’s code, print the saved server response and then exit accurately. Wrapping the assignment in ! would invert the status visible inside the branch.

Capture response metadata without mixing it into JSON

curl --silent --show-error   --output response.json   --dump-header response.headers   --write-out 'status=%{http_code} total=%{time_total} remote=%{remote_ip}
'   --json @payload.json   https://api.example.test/v1/orders

Response headers and body are different artifacts. Keep metrics on standard output or a separate file, and validate the claimed media type before passing the body to jq.

Keep authentication out of source and casual logs

curl --config - <<'CURL_CONFIG'
silent
show-error
fail-with-body
header = "Accept: application/json"
url = "https://api.example.test/v1/orders"
CURL_CONFIG

A bearer token passed as a command-line argument may be visible to other processes or captured in CI logs. Prefer the CI secret integration, a protected temporary curl config, short-lived workload identity, or a credential helper supported by the environment. Never use verbose tracing in production without redacting authorization and cookies.

Handle redirects deliberately

Do not add --location reflexively to authenticated POSTs. Review the redirect status, destination host and curl behavior. A misconfigured HTTP-to-HTTPS or cross-host redirect can change method behavior or risk credential exposure. Use the final HTTPS endpoint directly whenever possible.

Retry only when the operation is safe to repeat

curl --silent --show-error --fail-with-body   --retry 3   --retry-max-time 20   --header "Idempotency-Key: $REQUEST_ID"   --json @payload.json   https://api.example.test/v1/orders

curl warns that retrying all errors can duplicate data. A POST that creates an order needs an application-level idempotency contract; a random header has no effect unless the server stores and enforces it. Respect Retry-After, cap total time and distinguish connection failure from a definitive validation response.

Set time budgets

curl --connect-timeout 3   --max-time 15   --silent --show-error --fail-with-body   --json @payload.json   https://api.example.test/v1/orders

Without deadlines, automation can hold a worker indefinitely. Select budgets from service objectives and payload size, then surface whether the timeout occurred during name lookup, connection, TLS or transfer.

Test the complete response contract

jq -e '
  (.id | type == "string") and
  (.status == "created")
' response.json >/dev/null

HTTP 201 is not enough if the body is truncated, belongs to another schema version or lacks the ID needed by the next step. Validate status, content type and minimum schema, and retain the server request ID for support.

ProblemControl
Wrong request media type--json or explicit JSON headers
Shell breaks JSON quotingGenerate with jq and send from file/stdin
HTTP 500 exits zero--fail-with-body
Error body lostSeparate output file plus preserved body
Duplicate create on retryServer-enforced idempotency and bounded retry

A successful TCP transfer is not a successful API operation. Automation must evaluate the HTTP contract and the response body it depends on.

Sources: the solved Stack Overflow question (question by kamaci; accepted answer by Sean Patrick Floyd, CC BY-SA), curl’s official JSON guide and command-line manual.

Primary source: Review the official reference ↗