Using the Aryn SDK - Aryn Documentation

Installation

We recommend installing the Aryn SDK library using pip:

pip install aryn-sdk

Partitioning a Document

Partition a document like so:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f)

partition_file takes the same options as curl, except as keyword arguments. You can find a list of options here.

Extract properties from a document

Create a schema by hand, or get help from DocParse Suggest Properties. Provide the schema in property_extraction_options like so:

from aryn_sdk.partition import partition_file

property_email = {
      "name": "vendor_email",
      "type": {
        "type": "string",
        "description": "Vendor's email address, if not provided, return null",
        "examples": ["billing@abcservices.com", "john@consulting.com"],
        "validators": [
          {
            "type": "regex",
            "regex": "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
          }
        ]
      }
    }
schema = {"properties": [property_email]}

with open("my_invoice.pdf", "rb") as f:
   data = partition_file(f, property_extraction_options={"schema": schema})

You can add "voting": True to the property_extraction_options dictionary to enable to enable voting across multiple LLMs.

Key management

By default, aryn-sdk looks for Aryn API keys first in the environment variable ARYN_API_KEY, and then in ~/.aryn/config.yaml. You can override this behavior by specifying a key directly or a different path to the Aryn config file:

from aryn_sdk.partition import partition_file
from aryn_sdk.config import ArynConfig
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, aryn_api_key="YOUR-API-KEY")
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, aryn_config=ArynConfig(aryn_config_path="~/dotfiles/.aryn/config.yaml"))

Helper Functions

aryn_sdk provides some helper functions to make working with and visualizing the output of partition_file easier.

from aryn_sdk.partition import partition_file, table_elem_to_dataframe, table_elem_to_html, draw_with_boxes
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, extract_table_structure=True, text_mode="standard_ocr", extract_images=True, threshold=0.35)

# Produce a pandas DataFrame representing one of the extracted tables
table_elements = [elt for elt in data['elements'] if elt['type'] == 'table']
dataframe = table_elem_to_dataframe(table_elements[0])

# Convert the first table into HTML.
html = table_elem_to_html(table_elements[0])

# Draw the detected bounding boxes on the pages. requires poppler
images = draw_with_boxes("mydocument.pdf", data)

Different File Formats

It is easy to process files with different formats using the aryn-sdk:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f)
with open("mydocument.docx", "rb") as f:
   data = partition_file(f)
with open("mypresentation.doc", "rb") as f:
   data = partition_file(f)
with open("mypresentation.pptx", "rb") as f:
   data = partition_file(f)
with open("mypresentation.ppt", "rb") as f:
   data = partition_file(f)

URLs as input

The Aryn SDK also supports document input via a file path or URL to a document hosted on a remote server:

from aryn_sdk.partition import partition_file
data = partition_file("/home/jdoe/Documents/Memo.pdf")
data = partition_file("file:///home/jdoe/Documents/Memo.pdf")
data = partition_file("https://www.example.com/assets/proposal.pdf")
data = partition_file("https://aryn-public.s3.amazonaws.com/partitioner-blog-data/crispr.pdf")

Chunking a document

Chunking support has been added in v0.1.9. You can enable the default chunking options by specifying an empty dict:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, chunking_options={})

Here is an example specifying certain chunking options:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, 
      chunking_options={
         "strategy": "context_rich",
         "tokenizer": "openai_tokenizer",
         "tokenizer_options": {
            "model_name": "text-embedding-3-small"
         },
         "merge_across_pages": True,
         "max_tokens": 512,
      }
   )

The full chunking options are documented here.

Asynchronous requests

If you need to submit a large number of partitioning requests at once, we recommend using the asynchronous version of the API: partition_file_async_submit. The asynchronous API submits a file partitioning task to Aryn and returns with its task_id. You can use the returned task_id to keep track of your request to partition the file using the partition_file_async_result. To learn more check out the documentation and the tutorial.

​Installation

​Partitioning a Document

​Extract properties from a document

​Key management

​Helper Functions

​Different File Formats

​URLs as input

​Chunking a document

​Asynchronous requests