​
Installation

We recommend installing the Aryn SDK library using pip:

pip install aryn-sdk

​
Partitioning a Document

Partition a document like so:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f)

partition_file takes the same options as curl, except as keyword arguments. You can find a list of options here.

​
Key management

By default, aryn-sdk looks for Aryn API keys first in the environment variable ARYN_API_KEY, and then in ~/.aryn/config.yaml. You can override this behavior by specifying a key directly or a different path to the Aryn config file:

from aryn_sdk.partition import partition_file
from aryn_sdk.config import ArynConfig
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, aryn_api_key="YOUR-API-KEY")
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, aryn_config=ArynConfig(aryn_config_path="~/dotfiles/.aryn/config.yaml"))

​
Helper Functions

aryn_sdk provides some helper functions to make working with and visualizing the output of partition_file easier.

from aryn_sdk.partition import partition_file, table_elem_to_dataframe, draw_with_boxes
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, extract_table_structure=True, text_mode="standard_ocr", extract_images=True, threshold=0.35)

# Produce a pandas DataFrame representing one of the extracted tables
table_elements = [elt for elt in data['elements'] if elt['type'] == 'table']
dataframe = table_elem_to_dataframe(table_elements[0])

# Draw the detected bounding boxes on the pages. requires poppler
images = draw_with_boxes("mydocument.pdf", data)

​
Different File Formats

It is easy to process files with different formats using the aryn-sdk:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f)
with open("mydocument.docx", "rb") as f:
   data = partition_file(f)
with open("mypresentation.doc", "rb") as f:
   data = partition_file(f)
with open("mypresentation.pptx", "rb") as f:
   data = partition_file(f)
with open("mypresentation.ppt", "rb") as f:
   data = partition_file(f)

​
Chunking a document

Chunking support has been added in v0.1.9. You can enable the default chunking options by specifying an empty dict:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, chunking_options={})

Here is an example specifying certain chunking options:

from aryn_sdk.partition import partition_file
with open("mydocument.pdf", "rb") as f:
   data = partition_file(f, 
      chunking_options={
         "strategy": "context_rich",
         "tokenizer": "openai_tokenizer",
         "tokenizer_options": {
            "model_name": "text-embedding-3-small"
         },
         "merge_across_pages": True,
         "max_tokens": 512,
      }
   )

The full chunking options are documented here.

​
Asynchronous requests

If you need to submit a large number of partitioning requests at once, we recommend using the asynchronous version of the API: partition_file_async_submit. The asynchronous API submits a file partitioning task to Aryn and returns with its task_id. You can use the returned task_id to keep track of your request to partition the file using the partition_file_async_result. To learn more check out the documentation and the tutorial.

Was this page helpful?

Suggest editsRaise issue