Upload Files to Data Engine

This recipe shows how to upload files directly to the Gloo AI Data Engine using the Upload Files API. You’ll learn to upload single files, batch multiple files, associate custom IDs, and add metadata to your content. Upload once. Search instantly. Build faster. The Upload Files API lets you send any file directly to Gloo for real-time processing and indexing. Within minutes, your content becomes searchable, context-aware, and ready for AI-driven interactions.

Prerequisites

Before starting, ensure you have:

A Gloo AI Studio account
Your API key from the API Credentials page
Your Publisher ID from your Gloo AI Studio account
Authentication setup - Complete the Authentication Tutorial first

The Upload Files API requires an API key for authentication. If you haven’t set up authentication yet, follow the Authentication Tutorial.

Understanding the Upload Files API

The Upload Files API allows you to upload documents that get processed and made available for search and AI interaction. The primary endpoint is: POST /ingestion/v2/files

Key Features

Direct File Upload: Send files via multipart/form-data
Multi-File Support: Upload multiple files in a single request
Automatic Processing: Files are parsed, indexed, and made searchable
Format Detection: Automatic detection and processing of various file types
Duplicate Detection: Identifies and reports duplicate content

Required Fields

The Upload Files API requires two form fields:

publisher_id: Your publisher ID from Gloo AI Studio
files: The file(s) to upload (use the field name files, not file)

Supported File Types

PDF documents
Microsoft Word (.doc, .docx)
Plain text (.txt)
Markdown (.md)
And more

Step 1: Basic Single File Upload

Let’s start with uploading a single file. This demonstrates the core API call with proper authentication.

curl -X POST https://platform.ai.gloo.com/ingestion/v2/files \
  -H "Authorization: Bearer $GLOO_API_KEY" \
  -F "publisher_id=$GLOO_PUBLISHER_ID" \
  -F "files=@/path/to/your/document.pdf"

import requests
import os
from dotenv import load_dotenv

load_dotenv()

# Get API key (see Authentication tutorial)
API_KEY = os.getenv("GLOO_API_KEY")
PUBLISHER_ID = os.getenv("GLOO_PUBLISHER_ID")
API_URL = "https://platform.ai.gloo.com/ingestion/v2/files"

def upload_single_file(file_path):
    """Upload a single file to the Data Engine."""
    headers = {
        "Authorization": f"Bearer {API_KEY}"
    }

    with open(file_path, 'rb') as f:
        files = {"files": (os.path.basename(file_path), f)}
        data = {"publisher_id": PUBLISHER_ID}
        response = requests.post(API_URL, headers=headers, files=files, data=data)

    response.raise_for_status()
    return response.json()

# Example usage
if __name__ == "__main__":
    result = upload_single_file("/path/to/your/document.pdf")
    print(f"Upload result: {result}")

const fs = require('fs');
const path = require('path');
const FormData = require('form-data');
const axios = require('axios');
require('dotenv').config();

const API_KEY = process.env.GLOO_API_KEY;
const PUBLISHER_ID = process.env.GLOO_PUBLISHER_ID;
const API_URL = "https://platform.ai.gloo.com/ingestion/v2/files";

async function uploadSingleFile(filePath) {
    const formData = new FormData();
    formData.append('files', fs.createReadStream(filePath));
    formData.append('publisher_id', PUBLISHER_ID);

    const response = await axios.post(API_URL, formData, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        }
    });

    return response.data;
}

// Example usage
uploadSingleFile('/path/to/your/document.pdf')
    .then(result => console.log('Upload result:', result))
    .catch(error => console.error('Upload failed:', error.message));

import * as fs from 'fs';
import * as path from 'path';
import FormData from 'form-data';
import axios from 'axios';
import * as dotenv from 'dotenv';

dotenv.config();

const API_KEY = process.env.GLOO_API_KEY;
const PUBLISHER_ID = process.env.GLOO_PUBLISHER_ID;
const API_URL = "https://platform.ai.gloo.com/ingestion/v2/files";

interface UploadResponse {
    success: boolean;
    message: string;
    ingesting: string[];
    duplicates: string[];
}

async function uploadSingleFile(filePath: string): Promise<UploadResponse> {
    const formData = new FormData();
    formData.append('files', fs.createReadStream(filePath));
    formData.append('publisher_id', PUBLISHER_ID);

    const response = await axios.post<UploadResponse>(API_URL, formData, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        }
    });

    return response.data;
}

// Example usage
uploadSingleFile('/path/to/your/document.pdf')
    .then(result => console.log('Upload result:', result))
    .catch(error => console.error('Upload failed:', error.message));

<?php
declare(strict_types=1);

require_once __DIR__ . '/vendor/autoload.php';

use Dotenv\Dotenv;

$dotenv = Dotenv::createImmutable(__DIR__);
$dotenv->load();

$API_KEY = $_ENV['GLOO_API_KEY'] ?? null;
$PUBLISHER_ID = $_ENV['GLOO_PUBLISHER_ID'] ?? null;
$API_URL = 'https://platform.ai.gloo.com/ingestion/v2/files';

if (!$API_KEY) {
    die("Error: GLOO_API_KEY environment variable is required\n");
}

function uploadSingleFile(string $filePath): array {
    global $API_KEY, $PUBLISHER_ID, $API_URL;

    $ch = curl_init();

    $postFields = [
        'files' => new CURLFile($filePath, mime_content_type($filePath), basename($filePath)),
        'publisher_id' => $PUBLISHER_ID
    ];

    curl_setopt($ch, CURLOPT_URL, $API_URL);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields);
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
        'Authorization: Bearer ' . $API_KEY
    ]);

    $result = curl_exec($ch);

    if (curl_errno($ch)) {
        throw new Exception('Upload failed: ' . curl_error($ch));
    }

    curl_close($ch);

    return json_decode($result, true);
}

// Example usage
$result = uploadSingleFile('/path/to/your/document.pdf');
echo "Upload result: " . json_encode($result, JSON_PRETTY_PRINT) . "\n";
?>

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"mime/multipart"
	"net/http"
	"os"
	"path/filepath"
	"time"

	"github.com/joho/godotenv"
)

var (
	apiKey string
	publisherID string
	apiURL      = "https://platform.ai.gloo.com/ingestion/v2/files"
)

type UploadResponse struct {
	Success    bool     `json:"success"`
	Message    string   `json:"message"`
	Ingesting  []string `json:"ingesting"`
	Duplicates []string `json:"duplicates"`
}

func init() {
	godotenv.Load()
	apiKey = os.Getenv("GLOO_API_KEY")
	publisherID = os.Getenv("GLOO_PUBLISHER_ID")
	if apiKey == "" {
		fmt.Println("Error: GLOO_API_KEY environment variable is required")
		os.Exit(1)
	}
}

func uploadSingleFile(filePath string) (*UploadResponse, error) {
	file, err := os.Open(filePath)
	if err != nil {
		return nil, err
	}
	defer file.Close()

	var body bytes.Buffer
	writer := multipart.NewWriter(&body)

	part, err := writer.CreateFormFile("files", filepath.Base(filePath))
	if err != nil {
		return nil, err
	}

	if _, err := io.Copy(part, file); err != nil {
		return nil, err
	}

	// Add publisher_id field
	if err := writer.WriteField("publisher_id", publisherID); err != nil {
		return nil, err
	}

	writer.Close()

	req, err := http.NewRequest("POST", apiURL, &body)
	if err != nil {
		return nil, err
	}

	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("Content-Type", writer.FormDataContentType())

	client := &http.Client{Timeout: 60 * time.Second}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	respBody, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}

	if resp.StatusCode >= 300 {
		return nil, fmt.Errorf("upload failed: %s", string(respBody))
	}

	var result UploadResponse
	json.Unmarshal(respBody, &result)
	return &result, nil
}

func main() {
	result, err := uploadSingleFile("/path/to/your/document.pdf")
	if err != nil {
		fmt.Printf("Upload failed: %v\n", err)
		return
	}
	fmt.Printf("Upload result: %+v\n", result)
}

import io.github.cdimascio.dotenv.Dotenv;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;

public class FileUploader {
    private static final Dotenv dotenv = Dotenv.configure().ignoreIfMissing().load();
    private static final String API_KEY = dotenv.get("GLOO_API_KEY");
    private static final String PUBLISHER_ID = dotenv.get("GLOO_PUBLISHER_ID");
    private static final String API_URL = "https://platform.ai.gloo.com/ingestion/v2/files";
    private static final HttpClient httpClient = HttpClient.newHttpClient();

    public static String uploadSingleFile(String filePath) throws IOException, InterruptedException {
        if (API_KEY == null || API_KEY.isEmpty()) {
            throw new IllegalStateException("GLOO_API_KEY environment variable is required");
        }

        Path path = Path.of(filePath);
        String boundary = UUID.randomUUID().toString();

        byte[] body = buildMultipartBody(boundary, path);

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(API_URL))
                .header("Authorization", "Bearer " + API_KEY)
                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
                .POST(HttpRequest.BodyPublishers.ofByteArray(body))
                .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());

        if (response.statusCode() >= 300) {
            throw new IOException("Upload failed: " + response.body());
        }

        return response.body();
    }

    private static byte[] buildMultipartBody(String boundary, Path filePath) throws IOException {
        String fileName = filePath.getFileName().toString();
        byte[] fileContent = Files.readAllBytes(filePath);

        ByteArrayOutputStream out = new ByteArrayOutputStream();

        // Add publisher_id field
        out.write(("--" + boundary + "\r\n").getBytes(StandardCharsets.UTF_8));
        out.write("Content-Disposition: form-data; name=\"publisher_id\"\r\n\r\n".getBytes(StandardCharsets.UTF_8));
        out.write((PUBLISHER_ID + "\r\n").getBytes(StandardCharsets.UTF_8));

        // Add file field
        out.write(("--" + boundary + "\r\n").getBytes(StandardCharsets.UTF_8));
        out.write(("Content-Disposition: form-data; name=\"files\"; filename=\"" + fileName + "\"\r\n").getBytes(StandardCharsets.UTF_8));
        out.write("Content-Type: application/octet-stream\r\n\r\n".getBytes(StandardCharsets.UTF_8));
        out.write(fileContent);
        out.write(("\r\n--" + boundary + "--\r\n").getBytes(StandardCharsets.UTF_8));

        return out.toByteArray();
    }

    public static void main(String[] args) {
        try {
            String result = uploadSingleFile("/path/to/your/document.pdf");
            System.out.println("Upload result: " + result);
        } catch (Exception e) {
            System.err.println("Upload failed: " + e.getMessage());
        }
    }
}

Expected Response

{
    "success": true,
    "message": "File processing started in background.",
    "ingesting": [
        "c999008e-de60-495c-8c9f-6a4b59cdb04b"
    ],
    "duplicates": []
}

Step 2: Multi-File Upload

You can upload multiple files in a single request for better efficiency.

You can mix file types freely - the endpoint automatically detects and processes each format.

curl -X POST https://platform.ai.gloo.com/ingestion/v2/files \
  -H "Authorization: Bearer $GLOO_API_KEY" \
  -F "publisher_id=$GLOO_PUBLISHER_ID" \
  -F "files=@/path/to/document1.pdf" \
  -F "files=@/path/to/document2.pdf" \
  -F "files=@/path/to/document3.docx"

def upload_multiple_files(file_paths):
    """Upload multiple files to the Data Engine."""
    headers = {
        "Authorization": f"Bearer {API_KEY}"
    }

    files = []
    open_files = []

    try:
        for file_path in file_paths:
            f = open(file_path, 'rb')
            open_files.append(f)
            files.append(('files', (os.path.basename(file_path), f)))

        data = {"publisher_id": PUBLISHER_ID}
        response = requests.post(API_URL, headers=headers, files=files, data=data)
        response.raise_for_status()
        return response.json()
    finally:
        for f in open_files:
            f.close()

# Example usage
if __name__ == "__main__":
    files_to_upload = [
        "/path/to/document1.pdf",
        "/path/to/document2.pdf",
        "/path/to/document3.docx"
    ]
    result = upload_multiple_files(files_to_upload)
    print(f"Uploaded {len(result.get('ingesting', []))} files")
    if result.get('duplicates'):
        print(f"Duplicates found: {len(result['duplicates'])}")

async function uploadMultipleFiles(filePaths) {
    const formData = new FormData();

    for (const filePath of filePaths) {
        formData.append('files', fs.createReadStream(filePath));
    }
    formData.append('publisher_id', PUBLISHER_ID);

    const response = await axios.post(API_URL, formData, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        }
    });

    return response.data;
}

// Example usage
const filesToUpload = [
    '/path/to/document1.pdf',
    '/path/to/document2.pdf',
    '/path/to/document3.docx'
];

uploadMultipleFiles(filesToUpload)
    .then(result => {
        console.log(`Uploaded ${result.ingesting?.length || 0} files`);
        if (result.duplicates?.length) {
            console.log(`Duplicates found: ${result.duplicates.length}`);
        }
    })
    .catch(error => console.error('Upload failed:', error.message));

async function uploadMultipleFiles(filePaths: string[]): Promise<UploadResponse> {
    const formData = new FormData();

    for (const filePath of filePaths) {
        formData.append('files', fs.createReadStream(filePath));
    }
    formData.append('publisher_id', PUBLISHER_ID);

    const response = await axios.post<UploadResponse>(API_URL, formData, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        }
    });

    return response.data;
}

// Example usage
const filesToUpload = [
    '/path/to/document1.pdf',
    '/path/to/document2.pdf',
    '/path/to/document3.docx'
];

uploadMultipleFiles(filesToUpload)
    .then(result => {
        console.log(`Uploaded ${result.ingesting?.length || 0} files`);
        if (result.duplicates?.length) {
            console.log(`Duplicates found: ${result.duplicates.length}`);
        }
    })
    .catch(error => console.error('Upload failed:', error.message));

function uploadMultipleFiles(array $filePaths): array {
    global $API_KEY, $PUBLISHER_ID, $API_URL;

    $ch = curl_init();

    $postFields = ['publisher_id' => $PUBLISHER_ID];
    foreach ($filePaths as $index => $filePath) {
        $postFields["files[$index]"] = new CURLFile(
            $filePath,
            mime_content_type($filePath),
            basename($filePath)
        );
    }

    curl_setopt($ch, CURLOPT_URL, $API_URL);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields);
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
        'Authorization: Bearer ' . $API_KEY
    ]);

    $result = curl_exec($ch);

    if (curl_errno($ch)) {
        throw new Exception('Upload failed: ' . curl_error($ch));
    }

    curl_close($ch);

    return json_decode($result, true);
}

// Example usage
$filesToUpload = [
    '/path/to/document1.pdf',
    '/path/to/document2.pdf',
    '/path/to/document3.docx'
];

$result = uploadMultipleFiles($filesToUpload);
echo "Uploaded " . count($result['ingesting'] ?? []) . " files\n";
if (!empty($result['duplicates'])) {
    echo "Duplicates found: " . count($result['duplicates']) . "\n";
}

func uploadMultipleFiles(filePaths []string) (*UploadResponse, error) {
	var body bytes.Buffer
	writer := multipart.NewWriter(&body)

	for _, filePath := range filePaths {
		file, err := os.Open(filePath)
		if err != nil {
			return nil, err
		}

		part, err := writer.CreateFormFile("files", filepath.Base(filePath))
		if err != nil {
			file.Close()
			return nil, err
		}

		if _, err := io.Copy(part, file); err != nil {
			file.Close()
			return nil, err
		}
		file.Close()
	}

	// Add publisher_id field
	if err := writer.WriteField("publisher_id", publisherID); err != nil {
		return nil, err
	}

	writer.Close()

	req, err := http.NewRequest("POST", apiURL, &body)
	if err != nil {
		return nil, err
	}

	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("Content-Type", writer.FormDataContentType())

	client := &http.Client{Timeout: 120 * time.Second}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	respBody, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}

	if resp.StatusCode >= 300 {
		return nil, fmt.Errorf("upload failed: %s", string(respBody))
	}

	var result UploadResponse
	json.Unmarshal(respBody, &result)
	return &result, nil
}

func main() {
	filesToUpload := []string{
		"/path/to/document1.pdf",
		"/path/to/document2.pdf",
		"/path/to/document3.docx",
	}

	result, err := uploadMultipleFiles(filesToUpload)
	if err != nil {
		fmt.Printf("Upload failed: %v\n", err)
		return
	}

	fmt.Printf("Uploaded %d files\n", len(result.Ingesting))
	if len(result.Duplicates) > 0 {
		fmt.Printf("Duplicates found: %d\n", len(result.Duplicates))
	}
}

public static String uploadMultipleFiles(String[] filePaths) throws IOException, InterruptedException {
    if (API_KEY == null || API_KEY.isEmpty()) {
        throw new IllegalStateException("GLOO_API_KEY environment variable is required");
    }

    String boundary = UUID.randomUUID().toString();
    StringBuilder body = new StringBuilder();

    // Add publisher_id field
    body.append("--").append(boundary).append("\r\n");
    body.append("Content-Disposition: form-data; name=\"publisher_id\"\r\n\r\n");
    body.append(PUBLISHER_ID).append("\r\n");

    for (String filePath : filePaths) {
        Path path = Path.of(filePath);
        String fileName = path.getFileName().toString();
        byte[] fileContent = Files.readAllBytes(path);

        body.append("--").append(boundary).append("\r\n");
        body.append("Content-Disposition: form-data; name=\"files\"; filename=\"")
            .append(fileName).append("\"\r\n");
        body.append("Content-Type: application/octet-stream\r\n\r\n");
        body.append(new String(fileContent));
        body.append("\r\n");
    }
    body.append("--").append(boundary).append("--\r\n");

    HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(API_URL))
            .header("Authorization", "Bearer " + API_KEY)
            .header("Content-Type", "multipart/form-data; boundary=" + boundary)
            .POST(HttpRequest.BodyPublishers.ofString(body.toString()))
            .build();

    HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());

    if (response.statusCode() >= 300) {
        throw new IOException("Upload failed: " + response.body());
    }

    return response.body();
}

public static void main(String[] args) {
    try {
        String[] filesToUpload = {
            "/path/to/document1.pdf",
            "/path/to/document2.pdf",
            "/path/to/document3.docx"
        };
        String result = uploadMultipleFiles(filesToUpload);
        System.out.println("Upload result: " + result);
    } catch (Exception e) {
        System.err.println("Upload failed: " + e.getMessage());
    }
}

Expected Response

{
    "success": true,
    "message": "File processing started in background.",
    "ingesting": [
        "c999008e-de60-495c-8c9f-6a4b59cdb04b",
        "b10e85d8-243d-46e2-9504-d93874a9ebcb",
        "b45058a8-2f8c-4a88-8aba-7adb4afcd38d"
    ],
    "duplicates": []
}

Step 3: Using Producer ID

The producer_id query parameter lets you associate your internal ID with an uploaded file. This is useful for tracking and updating content from your system.

If the producer_id field is supplied with a multi-file upload, the ID will be ignored. Producer IDs must have a one-to-one relationship with a file.

curl -X POST "https://platform.ai.gloo.com/ingestion/v2/files?producer_id=my-internal-id-12345" \
  -H "Authorization: Bearer $GLOO_API_KEY" \
  -F "publisher_id=$GLOO_PUBLISHER_ID" \
  -F "files=@/path/to/your/document.pdf"

def upload_file_with_producer_id(file_path, producer_id):
    """Upload a file with a custom producer ID."""
    headers = {
        "Authorization": f"Bearer {API_KEY}"
    }

    params = {
        "producer_id": producer_id
    }

    with open(file_path, 'rb') as f:
        files = {"files": (os.path.basename(file_path), f)}
        data = {"publisher_id": PUBLISHER_ID}
        response = requests.post(API_URL, headers=headers, files=files, data=data, params=params)

    response.raise_for_status()
    return response.json()

# Example usage
if __name__ == "__main__":
    result = upload_file_with_producer_id(
        "/path/to/document.pdf",
        "my-internal-id-12345"
    )
    print(f"Upload result: {result}")

async function uploadFileWithProducerId(filePath, producerId) {
    const formData = new FormData();
    formData.append('files', fs.createReadStream(filePath));
    formData.append('publisher_id', PUBLISHER_ID);

    const response = await axios.post(API_URL, formData, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        },
        params: {
            producer_id: producerId
        }
    });

    return response.data;
}

// Example usage
uploadFileWithProducerId('/path/to/document.pdf', 'my-internal-id-12345')
    .then(result => console.log('Upload result:', result))
    .catch(error => console.error('Upload failed:', error.message));

async function uploadFileWithProducerId(filePath: string, producerId: string): Promise<UploadResponse> {
    const formData = new FormData();
    formData.append('files', fs.createReadStream(filePath));
    formData.append('publisher_id', PUBLISHER_ID);

    const response = await axios.post<UploadResponse>(API_URL, formData, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        },
        params: {
            producer_id: producerId
        }
    });

    return response.data;
}

// Example usage
uploadFileWithProducerId('/path/to/document.pdf', 'my-internal-id-12345')
    .then(result => console.log('Upload result:', result))
    .catch(error => console.error('Upload failed:', error.message));

function uploadFileWithProducerId(string $filePath, string $producerId): array {
    global $API_KEY, $PUBLISHER_ID, $API_URL;

    $url = $API_URL . '?' . http_build_query(['producer_id' => $producerId]);

    $ch = curl_init();

    $postFields = [
        'files' => new CURLFile($filePath, mime_content_type($filePath), basename($filePath)),
        'publisher_id' => $PUBLISHER_ID
    ];

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields);
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
        'Authorization: Bearer ' . $API_KEY
    ]);

    $result = curl_exec($ch);

    if (curl_errno($ch)) {
        throw new Exception('Upload failed: ' . curl_error($ch));
    }

    curl_close($ch);

    return json_decode($result, true);
}

// Example usage
$result = uploadFileWithProducerId('/path/to/document.pdf', 'my-internal-id-12345');
echo "Upload result: " . json_encode($result, JSON_PRETTY_PRINT) . "\n";

func uploadFileWithProducerId(filePath, producerId string) (*UploadResponse, error) {
	file, err := os.Open(filePath)
	if err != nil {
		return nil, err
	}
	defer file.Close()

	var body bytes.Buffer
	writer := multipart.NewWriter(&body)

	part, err := writer.CreateFormFile("files", filepath.Base(filePath))
	if err != nil {
		return nil, err
	}

	if _, err := io.Copy(part, file); err != nil {
		return nil, err
	}

	// Add publisher_id field
	if err := writer.WriteField("publisher_id", publisherID); err != nil {
		return nil, err
	}

	writer.Close()

	url := fmt.Sprintf("%s?producer_id=%s", apiURL, producerId)

	req, err := http.NewRequest("POST", url, &body)
	if err != nil {
		return nil, err
	}

	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("Content-Type", writer.FormDataContentType())

	client := &http.Client{Timeout: 60 * time.Second}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	respBody, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}

	if resp.StatusCode >= 300 {
		return nil, fmt.Errorf("upload failed: %s", string(respBody))
	}

	var result UploadResponse
	json.Unmarshal(respBody, &result)
	return &result, nil
}

func main() {
	result, err := uploadFileWithProducerId("/path/to/document.pdf", "my-internal-id-12345")
	if err != nil {
		fmt.Printf("Upload failed: %v\n", err)
		return
	}
	fmt.Printf("Upload result: %+v\n", result)
}

public static String uploadFileWithProducerId(String filePath, String producerId)
        throws IOException, InterruptedException {
    if (API_KEY == null || API_KEY.isEmpty()) {
        throw new IllegalStateException("GLOO_API_KEY environment variable is required");
    }

    Path path = Path.of(filePath);
    String boundary = UUID.randomUUID().toString();
    String body = buildMultipartBodyWithPublisher(boundary, path);

    String url = API_URL + "?producer_id=" + producerId;

    HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Authorization", "Bearer " + API_KEY)
            .header("Content-Type", "multipart/form-data; boundary=" + boundary)
            .POST(HttpRequest.BodyPublishers.ofString(body))
            .build();

    HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());

    if (response.statusCode() >= 300) {
        throw new IOException("Upload failed: " + response.body());
    }

    return response.body();
}

// Note: buildMultipartBodyWithPublisher includes publisher_id and uses "files" field name
// See the complete example in the sandbox code for full implementation

public static void main(String[] args) {
    try {
        String result = uploadFileWithProducerId("/path/to/document.pdf", "my-internal-id-12345");
        System.out.println("Upload result: " + result);
    } catch (Exception e) {
        System.err.println("Upload failed: " + e.getMessage());
    }
}

Step 4: Adding Metadata to Uploaded Content

After uploading files, you can add or update metadata using the Update Item Metadata endpoint. Identify content by either Gloo’s item_id (returned from the upload) or your producer_id.

Only the fields you include are updated; omitted fields remain unchanged.

curl -X POST https://platform.ai.gloo.com/engine/v2/item \
  -H "Authorization: Bearer $GLOO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "publisher_id": "your-publisher-id",
    "item_id": "c999008e-de60-495c-8c9f-6a4b59cdb04b",
    "item_title": "Document Title",
    "item_subtitle": "A brief subtitle",
    "author": ["John Doe", "Jane Smith"],
    "publication_date": "2025-01-15",
    "item_tags": ["category1", "category2"]
  }'

METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item"

def update_item_metadata(item_id=None, producer_id=None, publisher_id=None, **metadata):
    """Update metadata for an uploaded item."""
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }

    data = {
        "publisher_id": publisher_id
    }

    if item_id:
        data["item_id"] = item_id
    if producer_id:
        data["producer_id"] = producer_id

    data.update(metadata)

    response = requests.post(METADATA_URL, headers=headers, json=data)
    response.raise_for_status()
    return response.json()

# Example usage
if __name__ == "__main__":
    result = update_item_metadata(
        item_id="c999008e-de60-495c-8c9f-6a4b59cdb04b",
        publisher_id="your-publisher-id",
        item_title="Document Title",
        item_subtitle="A brief subtitle",
        author=["John Doe", "Jane Smith"],
        publication_date="2025-01-15",
        item_tags=["category1", "category2"]
    )
    print(f"Metadata update result: {result}")

const METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item";

async function updateItemMetadata({ itemId, producerId, publisherId, ...metadata }) {
    const data = {
        publisher_id: publisherId,
        ...(itemId && { item_id: itemId }),
        ...(producerId && { producer_id: producerId }),
        ...metadata
    };

    const response = await axios.post(METADATA_URL, data, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            'Content-Type': 'application/json'
        }
    });

    return response.data;
}

// Example usage
updateItemMetadata({
    itemId: 'c999008e-de60-495c-8c9f-6a4b59cdb04b',
    publisherId: 'your-publisher-id',
    item_title: 'Document Title',
    item_subtitle: 'A brief subtitle',
    author: ['John Doe', 'Jane Smith'],
    publication_date: '2025-01-15',
    item_tags: ['category1', 'category2']
})
    .then(result => console.log('Metadata update result:', result))
    .catch(error => console.error('Update failed:', error.message));

const METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item";

interface ItemMetadata {
    itemId?: string;
    producerId?: string;
    publisherId: string;
    item_title?: string;
    item_subtitle?: string;
    author?: string[];
    publication_date?: string;
    item_tags?: string[];
    item_summary?: string;
    item_url?: string;
    item_image?: string;
}

interface MetadataResponse {
    success: boolean;
    message: string;
}

async function updateItemMetadata(metadata: ItemMetadata): Promise<MetadataResponse> {
    const { itemId, producerId, publisherId, ...rest } = metadata;

    const data = {
        publisher_id: publisherId,
        ...(itemId && { item_id: itemId }),
        ...(producerId && { producer_id: producerId }),
        ...rest
    };

    const response = await axios.post<MetadataResponse>(METADATA_URL, data, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            'Content-Type': 'application/json'
        }
    });

    return response.data;
}

// Example usage
updateItemMetadata({
    itemId: 'c999008e-de60-495c-8c9f-6a4b59cdb04b',
    publisherId: 'your-publisher-id',
    item_title: 'Document Title',
    item_subtitle: 'A brief subtitle',
    author: ['John Doe', 'Jane Smith'],
    publication_date: '2025-01-15',
    item_tags: ['category1', 'category2']
})
    .then(result => console.log('Metadata update result:', result))
    .catch(error => console.error('Update failed:', error.message));

$METADATA_URL = 'https://platform.ai.gloo.com/engine/v2/item';

function updateItemMetadata(array $params): array {
    global $API_KEY, $METADATA_URL;

    $ch = curl_init();

    curl_setopt($ch, CURLOPT_URL, $METADATA_URL);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
        'Authorization: Bearer ' . $API_KEY,
        'Content-Type: application/json'
    ]);

    $result = curl_exec($ch);

    if (curl_errno($ch)) {
        throw new Exception('Update failed: ' . curl_error($ch));
    }

    curl_close($ch);

    return json_decode($result, true);
}

// Example usage
$result = updateItemMetadata([
    'publisher_id' => 'your-publisher-id',
    'item_id' => 'c999008e-de60-495c-8c9f-6a4b59cdb04b',
    'item_title' => 'Document Title',
    'item_subtitle' => 'A brief subtitle',
    'author' => ['John Doe', 'Jane Smith'],
    'publication_date' => '2025-01-15',
    'item_tags' => ['category1', 'category2']
]);
echo "Metadata update result: " . json_encode($result, JSON_PRETTY_PRINT) . "\n";

var metadataURL = "https://platform.ai.gloo.com/engine/v2/item"

type ItemMetadata struct {
	PublisherID     string   `json:"publisher_id"`
	ItemID          string   `json:"item_id,omitempty"`
	ProducerID      string   `json:"producer_id,omitempty"`
	ItemTitle       string   `json:"item_title,omitempty"`
	ItemSubtitle    string   `json:"item_subtitle,omitempty"`
	Author          []string `json:"author,omitempty"`
	PublicationDate string   `json:"publication_date,omitempty"`
	ItemTags        []string `json:"item_tags,omitempty"`
}

type MetadataResponse struct {
	Success bool   `json:"success"`
	Message string `json:"message"`
}

func updateItemMetadata(metadata ItemMetadata) (*MetadataResponse, error) {
	jsonData, err := json.Marshal(metadata)
	if err != nil {
		return nil, err
	}

	req, err := http.NewRequest("POST", metadataURL, bytes.NewBuffer(jsonData))
	if err != nil {
		return nil, err
	}

	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("Content-Type", "application/json")

	client := &http.Client{Timeout: 30 * time.Second}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	respBody, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}

	if resp.StatusCode >= 300 {
		return nil, fmt.Errorf("update failed: %s", string(respBody))
	}

	var result MetadataResponse
	json.Unmarshal(respBody, &result)
	return &result, nil
}

func main() {
	result, err := updateItemMetadata(ItemMetadata{
		PublisherID:     "your-publisher-id",
		ItemID:          "c999008e-de60-495c-8c9f-6a4b59cdb04b",
		ItemTitle:       "Document Title",
		ItemSubtitle:    "A brief subtitle",
		Author:          []string{"John Doe", "Jane Smith"},
		PublicationDate: "2025-01-15",
		ItemTags:        []string{"category1", "category2"},
	})
	if err != nil {
		fmt.Printf("Update failed: %v\n", err)
		return
	}
	fmt.Printf("Metadata update result: %+v\n", result)
}

private static final String METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item";

public static String updateItemMetadata(String jsonPayload) throws IOException, InterruptedException {
    if (API_KEY == null || API_KEY.isEmpty()) {
        throw new IllegalStateException("GLOO_API_KEY environment variable is required");
    }

    HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(METADATA_URL))
            .header("Authorization", "Bearer " + API_KEY)
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonPayload))
            .build();

    HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());

    if (response.statusCode() >= 300) {
        throw new IOException("Update failed: " + response.body());
    }

    return response.body();
}

public static void main(String[] args) {
    try {
        String payload = """
            {
                "publisher_id": "your-publisher-id",
                "item_id": "c999008e-de60-495c-8c9f-6a4b59cdb04b",
                "item_title": "Document Title",
                "item_subtitle": "A brief subtitle",
                "author": ["John Doe", "Jane Smith"],
                "publication_date": "2025-01-15",
                "item_tags": ["category1", "category2"]
            }
            """;
        String result = updateItemMetadata(payload);
        System.out.println("Metadata update result: " + result);
    } catch (Exception e) {
        System.err.println("Update failed: " + e.getMessage());
    }
}

Available Metadata Fields

Field	Type	Description
`publisher_id`	string	Required. Your publisher ID
`item_id`	string	Gloo’s item ID (from upload response)
`producer_id`	string	Your internal ID
`item_title`	string	Document title
`item_subtitle`	string	Document subtitle
`file_name`	string	Original filename
`publication_date`	string	Publication date (YYYY-MM-DD)
`item_image`	string	Image URL
`item_url`	string	Source URL
`item_summary`	string	Brief summary
`author`	string[]	List of authors
`item_tags`	string[]	Categorization tags

If you provide both item_id and producer_id, the service will prefer item_id when loading the target item. This allows for the producer_id to be updated.

Step 5: Verifying Your Upload

After uploading content, you can verify it in Gloo AI Studio:

Log in to Gloo AI Studio
Navigate to the Data Engine section from the main sidebar
Click on Your Data

You’ll see your uploaded files with their processing status and metadata.

Complete Example

Here’s a complete example that combines authentication, file upload, and metadata update:

"""
Complete file upload workflow for Gloo AI Data Engine.
"""
import requests
import time
import os
from dotenv import load_dotenv

load_dotenv()

# Configuration
API_KEY = os.getenv("GLOO_API_KEY")
PUBLISHER_ID = os.getenv("GLOO_PUBLISHER_ID")

UPLOAD_URL = "https://platform.ai.gloo.com/ingestion/v2/files"
METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item"

def upload_file(file_path, producer_id=None):
    """Upload a file to the Data Engine."""
    headers = {"Authorization": f"Bearer {API_KEY}"}
    params = {"producer_id": producer_id} if producer_id else {}

    with open(file_path, 'rb') as f:
        files = {"files": (os.path.basename(file_path), f)}
        data = {"publisher_id": PUBLISHER_ID}
        response = requests.post(UPLOAD_URL, headers=headers, files=files, data=data, params=params)

    response.raise_for_status()
    return response.json()

def update_metadata(item_id, **metadata):
    """Update metadata for an uploaded item."""
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }

    data = {"publisher_id": PUBLISHER_ID, "item_id": item_id, **metadata}
    response = requests.post(METADATA_URL, headers=headers, json=data)
    response.raise_for_status()
    return response.json()

def main():
    """Main workflow: upload file and add metadata."""
    file_path = "/path/to/your/document.pdf"

    # Step 1: Upload the file
    print(f"Uploading {file_path}...")
    upload_result = upload_file(file_path, producer_id="my-doc-001")
    print(f"Upload result: {upload_result}")

    if upload_result.get('ingesting'):
        item_id = upload_result['ingesting'][0]

        # Step 2: Add metadata
        print(f"Adding metadata to item {item_id}...")
        metadata_result = update_metadata(
            item_id,
            item_title="My Document Title",
            author=["Author Name"],
            publication_date="2025-01-15",
            item_tags=["documentation", "api"]
        )
        print(f"Metadata result: {metadata_result}")

if __name__ == "__main__":
    main()

/**
 * Complete file upload workflow for Gloo AI Data Engine.
 */
const fs = require('fs');
const path = require('path');
const FormData = require('form-data');
const axios = require('axios');
require('dotenv').config();

// Configuration
const API_KEY = process.env.GLOO_API_KEY;
const PUBLISHER_ID = process.env.GLOO_PUBLISHER_ID;

if (!API_KEY) {
    console.error("Error: GLOO_API_KEY environment variable is required");
    process.exit(1);
}

const UPLOAD_URL = "https://platform.ai.gloo.com/ingestion/v2/files";
const METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item";

async function uploadFile(filePath, producerId = null) {
    const formData = new FormData();
    formData.append('files', fs.createReadStream(filePath));
    formData.append('publisher_id', PUBLISHER_ID);

    const config = {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        }
    };

    if (producerId) {
        config.params = { producer_id: producerId };
    }

    const response = await axios.post(UPLOAD_URL, formData, config);
    return response.data;
}

async function updateMetadata(itemId, metadata) {
    const data = {
        publisher_id: PUBLISHER_ID,
        item_id: itemId,
        ...metadata
    };

    const response = await axios.post(METADATA_URL, data, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            'Content-Type': 'application/json'
        }
    });

    return response.data;
}

async function main() {
    const filePath = '/path/to/your/document.pdf';

    try {
        // Step 1: Upload the file
        console.log(`Uploading ${filePath}...`);
        const uploadResult = await uploadFile(filePath, 'my-doc-001');
        console.log('Upload result:', uploadResult);

        if (uploadResult.ingesting?.length > 0) {
            const itemId = uploadResult.ingesting[0];

            // Step 2: Add metadata
            console.log(`Adding metadata to item ${itemId}...`);
            const metadataResult = await updateMetadata(itemId, {
                item_title: 'My Document Title',
                author: ['Author Name'],
                publication_date: '2025-01-15',
                item_tags: ['documentation', 'api']
            });
            console.log('Metadata result:', metadataResult);
        }
    } catch (error) {
        console.error('Error:', error.response?.data || error.message);
    }
}

main();

/**
 * Complete file upload workflow for Gloo AI Data Engine.
 */
import * as fs from 'fs';
import * as path from 'path';
import FormData from 'form-data';
import axios from 'axios';
import * as dotenv from 'dotenv';

dotenv.config();

// Configuration
const API_KEY = process.env.GLOO_API_KEY!;
const PUBLISHER_ID = process.env.GLOO_PUBLISHER_ID!;

if (!API_KEY) {
    console.error("Error: GLOO_API_KEY environment variable is required");
    process.exit(1);
}

const UPLOAD_URL = "https://platform.ai.gloo.com/ingestion/v2/files";
const METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item";

// Types
interface UploadResponse {
    success: boolean;
    message: string;
    ingesting: string[];
    duplicates: string[];
}

interface MetadataResponse {
    success: boolean;
    message: string;
}

async function uploadFile(filePath: string, producerId?: string): Promise<UploadResponse> {
    const formData = new FormData();
    formData.append('files', fs.createReadStream(filePath));
    formData.append('publisher_id', PUBLISHER_ID);

    const config: any = {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            ...formData.getHeaders()
        }
    };

    if (producerId) {
        config.params = { producer_id: producerId };
    }

    const response = await axios.post<UploadResponse>(UPLOAD_URL, formData, config);
    return response.data;
}

async function updateMetadata(itemId: string, metadata: Record<string, any>): Promise<MetadataResponse> {
    const data = {
        publisher_id: PUBLISHER_ID,
        item_id: itemId,
        ...metadata
    };

    const response = await axios.post<MetadataResponse>(METADATA_URL, data, {
        headers: {
            'Authorization': `Bearer ${API_KEY}`,
            'Content-Type': 'application/json'
        }
    });

    return response.data;
}

async function main(): Promise<void> {
    const filePath = '/path/to/your/document.pdf';

    try {
        // Step 1: Upload the file
        console.log(`Uploading ${filePath}...`);
        const uploadResult = await uploadFile(filePath, 'my-doc-001');
        console.log('Upload result:', uploadResult);

        if (uploadResult.ingesting?.length > 0) {
            const itemId = uploadResult.ingesting[0];

            // Step 2: Add metadata
            console.log(`Adding metadata to item ${itemId}...`);
            const metadataResult = await updateMetadata(itemId, {
                item_title: 'My Document Title',
                author: ['Author Name'],
                publication_date: '2025-01-15',
                item_tags: ['documentation', 'api']
            });
            console.log('Metadata result:', metadataResult);
        }
    } catch (error: any) {
        console.error('Error:', error.response?.data || error.message);
    }
}

main();

<?php
/**
 * Complete file upload workflow for Gloo AI Data Engine.
 */
declare(strict_types=1);

require_once __DIR__ . '/vendor/autoload.php';

use Dotenv\Dotenv;

$dotenv = Dotenv::createImmutable(__DIR__);
$dotenv->safeLoad();

// Configuration
$API_KEY = $_ENV['GLOO_API_KEY'] ?? '';
$PUBLISHER_ID = $_ENV['GLOO_PUBLISHER_ID'] ?? '';

if (empty($API_KEY)) {
    die("Error: GLOO_API_KEY environment variable is required\n");
}

$UPLOAD_URL = 'https://platform.ai.gloo.com/ingestion/v2/files';
$METADATA_URL = 'https://platform.ai.gloo.com/engine/v2/item';

function uploadFile(string $filePath, ?string $producerId = null): array {
    global $API_KEY, $UPLOAD_URL, $PUBLISHER_ID;

    $url = $UPLOAD_URL;
    if ($producerId) {
        $url .= '?' . http_build_query(['producer_id' => $producerId]);
    }

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, [
        'files' => new CURLFile($filePath, mime_content_type($filePath), basename($filePath)),
        'publisher_id' => $PUBLISHER_ID
    ]);
    curl_setopt($ch, CURLOPT_HTTPHEADER, ['Authorization: Bearer ' . $API_KEY]);

    $result = curl_exec($ch);
    curl_close($ch);

    return json_decode($result, true);
}

function updateMetadata(string $itemId, array $metadata): array {
    global $API_KEY, $METADATA_URL, $PUBLISHER_ID;

    $data = array_merge(['publisher_id' => $PUBLISHER_ID, 'item_id' => $itemId], $metadata);

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $METADATA_URL);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
        'Authorization: Bearer ' . $API_KEY,
        'Content-Type: application/json'
    ]);

    $result = curl_exec($ch);
    curl_close($ch);

    return json_decode($result, true);
}

// Main workflow
$filePath = '/path/to/your/document.pdf';

echo "Uploading $filePath...\n";
$uploadResult = uploadFile($filePath, 'my-doc-001');
echo "Upload result: " . json_encode($uploadResult) . "\n";

if (!empty($uploadResult['ingesting'])) {
    $itemId = $uploadResult['ingesting'][0];

    echo "Adding metadata to item $itemId...\n";
    $metadataResult = updateMetadata($itemId, [
        'item_title' => 'My Document Title',
        'author' => ['Author Name'],
        'publication_date' => '2025-01-15',
        'item_tags' => ['documentation', 'api']
    ]);
    echo "Metadata result: " . json_encode($metadataResult) . "\n";
}
?>

// Complete file upload workflow for Gloo AI Data Engine.
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"mime/multipart"
	"net/http"
	"net/url"
	"os"
	"path/filepath"

	"github.com/joho/godotenv"
)

// Configuration
var (
	apiKey      string
	publisherID string
	uploadURL   = "https://platform.ai.gloo.com/ingestion/v2/files"
	metadataURL = "https://platform.ai.gloo.com/engine/v2/item"
)

// Types
type UploadResponse struct {
	Success    bool     `json:"success"`
	Message    string   `json:"message"`
	Ingesting  []string `json:"ingesting"`
	Duplicates []string `json:"duplicates"`
}

type MetadataResponse struct {
	Success bool   `json:"success"`
	Message string `json:"message"`
}

func init() {
	godotenv.Load()
	apiKey = os.Getenv("GLOO_API_KEY")
	publisherID = os.Getenv("GLOO_PUBLISHER_ID")
	if apiKey == "" {
		fmt.Println("Error: GLOO_API_KEY environment variable is required")
		os.Exit(1)
	}
}

func uploadFile(filePath string, producerID string) (*UploadResponse, error) {
	file, _ := os.Open(filePath)
	defer file.Close()

	var body bytes.Buffer
	writer := multipart.NewWriter(&body)
	part, _ := writer.CreateFormFile("files", filepath.Base(filePath))
	io.Copy(part, file)
	writer.WriteField("publisher_id", publisherID)
	writer.Close()

	targetURL := uploadURL
	if producerID != "" {
		u, _ := url.Parse(uploadURL)
		q := u.Query()
		q.Set("producer_id", producerID)
		u.RawQuery = q.Encode()
		targetURL = u.String()
	}

	req, _ := http.NewRequest("POST", targetURL, &body)
	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("Content-Type", writer.FormDataContentType())

	resp, _ := http.DefaultClient.Do(req)
	defer resp.Body.Close()

	respBody, _ := io.ReadAll(resp.Body)
	var result UploadResponse
	json.Unmarshal(respBody, &result)
	return &result, nil
}

func updateMetadata(itemID string, metadata map[string]interface{}) (*MetadataResponse, error) {
	metadata["publisher_id"] = publisherID
	metadata["item_id"] = itemID
	jsonData, _ := json.Marshal(metadata)

	req, _ := http.NewRequest("POST", metadataURL, bytes.NewBuffer(jsonData))
	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("Content-Type", "application/json")

	resp, _ := http.DefaultClient.Do(req)
	defer resp.Body.Close()

	respBody, _ := io.ReadAll(resp.Body)
	var result MetadataResponse
	json.Unmarshal(respBody, &result)
	return &result, nil
}

func main() {
	filePath := "/path/to/your/document.pdf"

	fmt.Printf("Uploading %s...\n", filePath)
	uploadResult, _ := uploadFile(filePath, "my-doc-001")
	fmt.Printf("Upload result: %+v\n", uploadResult)

	if len(uploadResult.Ingesting) > 0 {
		itemID := uploadResult.Ingesting[0]

		fmt.Printf("Adding metadata to item %s...\n", itemID)
		metadataResult, _ := updateMetadata(itemID, map[string]interface{}{
			"item_title":       "My Document Title",
			"author":           []string{"Author Name"},
			"publication_date": "2025-01-15",
			"item_tags":        []string{"documentation", "api"},
		})
		fmt.Printf("Metadata result: %+v\n", metadataResult)
	}
}

/**
 * Complete file upload workflow for Gloo AI Data Engine.
 */
package com.gloo.tutorial.uploadfiles;

import io.github.cdimascio.dotenv.Dotenv;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

import java.io.*;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.util.*;

public class Main {
    private static final Dotenv dotenv = Dotenv.configure().ignoreIfMissing().load();
    private static final String API_KEY = dotenv.get("GLOO_API_KEY");
    private static final String PUBLISHER_ID = dotenv.get("GLOO_PUBLISHER_ID");

    private static final String UPLOAD_URL = "https://platform.ai.gloo.com/ingestion/v2/files";
    private static final String METADATA_URL = "https://platform.ai.gloo.com/engine/v2/item";

    private static final HttpClient httpClient = HttpClient.newHttpClient();
    private static final Gson gson = new Gson();

    private static Map<String, Object> uploadFile(String filePath, String producerId) throws Exception {
        if (API_KEY == null || API_KEY.isEmpty()) {
            throw new IllegalStateException("GLOO_API_KEY environment variable is required");
        }

        Path path = Path.of(filePath);
        String boundary = UUID.randomUUID().toString();

        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        PrintWriter writer = new PrintWriter(new OutputStreamWriter(baos, StandardCharsets.UTF_8), true);

        // Add publisher_id
        writer.append("--").append(boundary).append("\r\n");
        writer.append("Content-Disposition: form-data; name=\"publisher_id\"\r\n\r\n");
        writer.append(PUBLISHER_ID).append("\r\n");

        // Add file
        writer.append("--").append(boundary).append("\r\n");
        writer.append("Content-Disposition: form-data; name=\"files\"; filename=\"")
                .append(path.getFileName().toString()).append("\"\r\n");
        writer.append("Content-Type: application/octet-stream\r\n\r\n");
        writer.flush();
        baos.write(Files.readAllBytes(path));
        writer.append("\r\n--").append(boundary).append("--\r\n");
        writer.flush();

        String url = producerId != null ? UPLOAD_URL + "?producer_id=" + producerId : UPLOAD_URL;

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(url))
                .header("Authorization", "Bearer " + API_KEY)
                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
                .POST(HttpRequest.BodyPublishers.ofByteArray(baos.toByteArray()))
                .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
        return gson.fromJson(response.body(), Map.class);
    }

    private static Map<String, Object> updateMetadata(String itemId, Map<String, Object> metadata) throws Exception {
        if (API_KEY == null || API_KEY.isEmpty()) {
            throw new IllegalStateException("GLOO_API_KEY environment variable is required");
        }

        JsonObject data = new JsonObject();
        data.addProperty("publisher_id", PUBLISHER_ID);
        data.addProperty("item_id", itemId);
        for (Map.Entry<String, Object> entry : metadata.entrySet()) {
            if (entry.getValue() instanceof String) {
                data.addProperty(entry.getKey(), (String) entry.getValue());
            } else if (entry.getValue() instanceof List) {
                data.add(entry.getKey(), gson.toJsonTree(entry.getValue()));
            }
        }

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(METADATA_URL))
                .header("Authorization", "Bearer " + API_KEY)
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(gson.toJson(data)))
                .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
        return gson.fromJson(response.body(), Map.class);
    }

    public static void main(String[] args) {
        try {
            String filePath = "/path/to/your/document.pdf";

            System.out.println("Uploading " + filePath + "...");
            Map<String, Object> uploadResult = uploadFile(filePath, "my-doc-001");
            System.out.println("Upload result: " + uploadResult);

            List<String> ingesting = (List<String>) uploadResult.get("ingesting");
            if (ingesting != null && !ingesting.isEmpty()) {
                String itemId = ingesting.get(0);

                System.out.println("Adding metadata to item " + itemId + "...");
                Map<String, Object> metadata = new HashMap<>();
                metadata.put("item_title", "My Document Title");
                metadata.put("author", List.of("Author Name"));
                metadata.put("publication_date", "2025-01-15");
                metadata.put("item_tags", List.of("documentation", "api"));

                Map<String, Object> metadataResult = updateMetadata(itemId, metadata);
                System.out.println("Metadata result: " + metadataResult);
            }
        } catch (Exception e) {
            System.err.println("Error: " + e.getMessage());
        }
    }
}

Troubleshooting

Error: 401 Unauthorized

Cause: API key is incorrect or has been revoked. Solution: Verify your API key is correct and has not been revoked.

Error: 403 Forbidden

Cause: Insufficient permissions for the publisher. Solution: Verify you have access to the specified publisher in Gloo AI Studio.

Error: 413 Request Entity Too Large

Cause: File size exceeds the limit. Solution: Check the API limits and consider splitting large files.

Error: Duplicate detected

Cause: A file with identical content already exists. Solution: This is informational - the file won’t be processed again. Check the duplicates array in the response.

Working Code Sample

View Complete Code

Clone or browse the complete working examples for all 6 languages (JavaScript, TypeScript, Python, PHP, Go, Java) with setup instructions.

Next Steps

Now that you can upload files to the Data Engine, explore:

Search API - Query your uploaded content
Chat Integration - Use uploaded content in conversations
Manage Content - Manage and update your content
Upload Files API - For additional API information
Update Item Metadata API - For additional API information

Get Started

Completions

Content

Search & Recommendations

Legacy

Upload Files to Data Engine

Prerequisites

Understanding the Upload Files API

Key Features

Required Fields

Supported File Types

Step 1: Basic Single File Upload

Expected Response

Step 2: Multi-File Upload

Expected Response

Step 3: Using Producer ID

Step 4: Adding Metadata to Uploaded Content

Available Metadata Fields

Step 5: Verifying Your Upload

Complete Example

Troubleshooting

Error: 401 Unauthorized

Error: 403 Forbidden

Error: 413 Request Entity Too Large

Error: Duplicate detected

Working Code Sample

View Complete Code

Next Steps

​Prerequisites

​Understanding the Upload Files API

​Key Features

​Required Fields

​Supported File Types

​Step 1: Basic Single File Upload

​Expected Response

​Step 2: Multi-File Upload

​Expected Response

​Step 3: Using Producer ID

​Step 4: Adding Metadata to Uploaded Content

​Available Metadata Fields

​Step 5: Verifying Your Upload

​Complete Example

​Troubleshooting

​Error: 401 Unauthorized

​Error: 403 Forbidden

​Error: 413 Request Entity Too Large

​Error: Duplicate detected

​Working Code Sample

View Complete Code

​Next Steps

Prerequisites

Understanding the Upload Files API

Key Features

Required Fields

Supported File Types

Step 1: Basic Single File Upload

Expected Response

Step 2: Multi-File Upload

Expected Response

Step 3: Using Producer ID

Step 4: Adding Metadata to Uploaded Content

Available Metadata Fields

Step 5: Verifying Your Upload

Complete Example

Troubleshooting

Error: 401 Unauthorized

Error: 403 Forbidden

Error: 413 Request Entity Too Large

Error: Duplicate detected

Working Code Sample

Next Steps