File connector

The file connector lets you send jobs without hosting files in S3 or publishing temporary URLs. Upload files once, reference their IDs in a connector definition, and reuse them across multiple jobs.

When to use the file connector

You need to process documents or videos that only exist locally.
You want to avoid managing temporary URLs or granting the service access to your buckets.
You plan to upload once and run multiple jobs against the same inputs.

Workflow overview

Upload files and capture the returned IDs.
Create jobs with a connector payload that lists one or more file_ids.

Upload files

Before using the file connector, upload your files to get their IDs. See Upload files for details on supported formats.

Create jobs with uploaded files

The connector definition references the files you uploaded. Order is preserved, so the first file ID in the list becomes the first input identifier in job responses.

TypeScript
Python

string

required

Set to "file_upload".

string[]

required

Array of uploaded file ID strings. You must own every referenced ID.

import { DataIngestionClient } from "@trelent/data-ingestion";
import type { JobInput } from "@trelent/data-ingestion";

const client = new DataIngestionClient();

const job: JobInput = {
  connector: {
    type: "file_upload",
    file_ids: [
      "e2d4c3d2-41c0-4a6b-a387-580807fb5ad2",
      "b4c77b85-3ad4-4cb0-8d06-5ff8ff5df2ad",
    ],
  },
  output: {
    type: "s3-signed-url",
    expires_minutes: 120,
  },
};

const response = await client.submitJob(job);
console.log("Job ID:", response.job_id);

Use the FileUploadConnector class to define the connector.

list[UUID]

required

List of uploaded file UUIDs. You must own every referenced ID.

from uuid import UUID
from trelent_data_ingestion_sdk import DataIngestionClient, JobInput, FileUploadConnector, S3SignedUrlOutput

client = DataIngestionClient()

job = JobInput(
    connector=FileUploadConnector(file_ids=[
        UUID("e2d4c3d2-41c0-4a6b-a387-580807fb5ad2"),
        UUID("b4c77b85-3ad4-4cb0-8d06-5ff8ff5df2ad"),
    ]),
    output=S3SignedUrlOutput(expires_minutes=120),
)

response = client.submit_job(job)
print("Job ID:", response.job_id)

List available uploads

Use List files to see which file IDs remain valid before creating jobs. Need a full walkthrough? Follow the quickstart to upload a file and process it end-to-end.

Getting started

Connectors

Configuration

Outputs

Jobs

Files

Admin

When to use the file connector

Workflow overview

Upload files

Create jobs with uploaded files

List available uploads

​When to use the file connector

​Workflow overview

​Upload files

​Create jobs with uploaded files

​List available uploads

When to use the file connector

Workflow overview

Upload files

Create jobs with uploaded files

List available uploads