[TRI-1882] Python support

[matt-aitken]

It would be great to offer support for Python.

Python has better support for many Machine Learning uses cases:

• Many ML libraries are only Python, or more complete in Python
• Some services have ML models that only work from Python (e.g. OpenAI's whisper model)
• Data Science and Engineering tools like pandas

It would be even better if you could mix languages. Have files alongside each other that are different languages but can declare tasks.

_TRI-1882

[matt-aitken]

Cloudflare Workers transpiles languages into JavaScript.

• Cloudflare blog post: https://blog.cloudflare.com/cloudflare-workers-announces-broad-language-support
• Languages that can compile to JS: https://github.com/jashkenas/coffeescript/wiki/List-of-languages-that-compile-to-JS
• Transcrypt is used to compile Python to JS: https://www.transcrypt.org/docs/html/installation_use.html

[listenaddress]

Hey Matt, I'd love to use trigger.dev with Python! Any idea when this might happen?

[ologbonowiwi]

Having this would be great! I want to do a job to run https://github.com/joshiayush/inb automatically and trigger.dev would be a great option if it offers support for Python.

[MaleicAcid]

+1 really need this.

[montaign]

pls advise on timeline

[LeEricCH]

+1 would really enjoy that!

[chanmathew]

+1! Big use case for this is running python based crawlers and other AI inference tasks.

[tgirotto]

+1 it would be a great use case for gis applications as well :)

[nullmastermind]

+1

[archodev]

+1

[saschaseniuk]

https://feedback.trigger.dev/p/python-support-2

You can upvote for python support to make it happen 😍

[BaileySimrell]

+1 this would be awesome .. just upvoted!

[peter-hanzo]

Here is the list that I want to use in my project. Many of these libraries I find very useful in any project:

---

Asynchronous and HTTP Requests

• aiohttp: Asynchronous HTTP client and server for Python.
• async-timeout: Manage timeouts in asynchronous operations.
• httpx: Asynchronous HTTP requests, an alternative to aiohttp.
• websocket-client: For implementing WebSocket connections.

---

Data Processing

• pandas: Data analysis and manipulation (e.g., checklists, tasks, statuses).
• numpy: Mathematical computations.
• sqlalchemy: ORM for database operations and queries.
• pydantic: Data validation.
• pydantic_core: Core utilities for Pydantic.
• jsonschema: JSON structure validation.
• jsonschema_specifications: Specifications for JSON schema validation.
• pyyaml: Work with YAML for configurations and templates.
• pytz: Handle time zones.
• python-dateutil: Work with dates and time-related calculations.
• s3fs: Work with AWS S3 file systems.
• smart_open: Work with cloud-based files.

---

Text and Query Handling

• spaCy: NLP library for parsing queries, extracting keywords, and entities.
• nltk: Natural Language Processing (e.g., syntax analysis and tokenization).
• textblob: Simple tool for text analysis.
• regex: Regular expressions for advanced text parsing.
• pyparsing: Text parsing library.
• text-unidecode: Decode text.
• ujson: Fast serialization and deserialization of JSON.

---

Reporting and Visualization

• Pillow: Work with images (e.g., before/after photos).
• imageio: Load and save images.
• tabulate: Format data into tables.
• matplotlib: Create graphs and charts.
• reportlab / fpdf: Automatically generate PDFs for reports and results.
• jinja2: Generate HTML templates for reports.
• openpyxl: Save checklists or reports in Excel format.
• xlrd: Work with Excel files.
• xlwt: Create Excel 97/2000 files.
• asciitree: Generate tree-like text structures.

---

Parsing and Web Data

• html5lib: Parse HTML.
• soupsieve: Search within HTML documents.
• lxml: Parse XML/HTML.
• webencodings: Work with text encodings.

---

Infrastructure and Utilities

• docker: For deploying the project in containers.
• boto3: Work with AWS, including S3 and other services.
• logging: Standard library for detailed logging of events.
• cryptography: Tools for encrypting data.
• tomli: Read TOML files.

---

Additional

• charset-normalizer: Detect text encodings.
• nh3: Generate HTML.
• zipp: Work with archives.
• sqlparse: Parse SQL statements.
• xarray: Analyze multidimensional arrays.

[zvictor]

Please give the #1686 PR a try!
We need testers.