Create a job

Use one connector (source) and one output (delivery mechanism) to submit a job. You’ll receive a job_id that you can poll until results are ready.

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000"
}

TypeScript
Python

import { DataIngestionClient, DataIngestionConfig } from "@trelent/data-ingestion";
import type { JobInput } from "@trelent/data-ingestion";

const client = new DataIngestionClient();

const job: JobInput = {
  connector: {
    type: "url",
    urls: [
      "https://example.com/file.pdf",
    ],
  },
  output: {
    type: "s3-signed-url",
  },
};

const { job_id } = await client.submitJob(job);
console.log("Submitted job:", job_id);

You can also configure the client via environment variables: TRELENT_DATA_INGESTION_API_URL and TRELENT_DATA_INGESTION_API_TOKEN.

from trelent_data_ingestion_sdk.client import DataIngestionClient
from trelent_data_ingestion_sdk.config import SDKConfig
from trelent_data_ingestion_sdk.models import JobInput, UrlConnector, S3SignedUrlOutput

client = DataIngestionClient()

job = JobInput(
    connector=UrlConnector(type="url", urls=[
        "https://example.com/file.pdf",
    ]),
    output=S3SignedUrlOutput(type="s3-signed-url"),
)

resp = client.submit_job(job)
print("Submitted job:", resp.job_id)

You can configure via environment variables: DATA_INGESTION_API_URL and DATA_INGESTION_API_TOKEN.

S3 connector example (optional)

If your files are in S3, use the S3 connector. Each object key becomes the stable identifier echoed in responses.

TypeScript
Python

import type { JobInput } from "@trelent/data-ingestion";

const job: JobInput = {
  connector: {
    type: "s3",
    bucket_name: "my-docs-bucket",
    prefixes: [
      { prefix: "docs/", recursive: true },
      "videos/demo.mp4",
    ],
  },
  output: { type: "bucket", bucket_name: "example-output-bucket", prefix: "processed/" },
};

from trelent_data_ingestion_sdk.models import JobInput, S3Connector, BucketOutput, S3Prefix

job = JobInput(
    connector=S3Connector(
        type="s3",
        bucket_name="my-docs-bucket",
        prefixes=[S3Prefix(prefix="docs/", recursive=True), "videos/demo.mp4"],
    ),
    output=BucketOutput(type="bucket", bucket_name="example-output-bucket", prefix="processed/")
)

Getting started

Connectors

Configuration

Outputs

Jobs

Files

Admin

S3 connector example (optional)

​S3 connector example (optional)

S3 connector example (optional)