Invalid regex when running Phi4-mini-instruct-onnx using phi3-qa.py

[kimmywork]

Output

(workbench) ➜  workbench git:(master) ✗ python phi3-qa.py -m cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4 -e cpu --verbose
Loading model...
Model loaded
Traceback (most recent call last):
  File "/Users/qliu23/workspace/ml/workbench/phi3-qa.py", line 98, in <module>
    main(args)
  File "/Users/qliu23/workspace/ml/workbench/phi3-qa.py", line 20, in main
    tokenizer = og.Tokenizer(model)
                ^^^^^^^^^^^^^^^^^^^
RuntimeError: Invalid regex:  ?[^\s\p{L}\p{N}]+[\r\n/]*
The expression contained an invalid escaped character, or a trailing escape.

Environment:

Package                                  Version
---------------------------------------- --------------
absl-py                                  2.1.0
accelerate                               1.4.0
aiofiles                                 24.1.0
aiohappyeyeballs                         2.4.6
aiohttp                                  3.11.12
aiolimiter                               1.2.1
aiosignal                                1.3.2
alembic                                  1.14.1
annotated-types                          0.7.0
anthropic                                0.46.0
anyio                                    4.8.0
anytree                                  2.12.1
appnope                                  0.1.4
argon2-cffi                              23.1.0
argon2-cffi-bindings                     21.2.0
argparse                                 1.4.0
arrow                                    1.3.0
asgiref                                  3.8.1
asttokens                                2.4.1
astunparse                               1.6.3
async-lru                                2.0.4
attrs                                    25.1.0
autogen-agentchat                        0.4.7
autogen-core                             0.4.7
autogen-ext                              0.4.7
autogenstudio                            0.4.1.7
autograd                                 1.7.0
azure-ai-documentintelligence            1.0.0
azure-ai-inference                       1.0.0b9
azure-common                             1.1.28
azure-core                               1.32.0
azure-cosmos                             4.9.0
azure-identity                           1.20.0
azure-search-documents                   11.5.2
azure-storage-blob                       12.24.1
babel                                    2.17.0
backoff                                  2.2.1
bcrypt                                   4.2.1
beartype                                 0.18.5
beautifulsoup4                           4.13.3
binance-connector                        3.12.0
bioc                                     2.1
bleach                                   6.2.0
blis                                     1.2.0
boto3                                    1.36.25
botocore                                 1.36.25
browser-use                              0.1.37
build                                    1.2.2.post1
cachetools                               5.5.2
catalogue                                2.0.10
certifi                                  2025.1.31
cffi                                     1.17.1
chardet                                  5.2.0
charset-normalizer                       3.4.1
chroma-hnswlib                           0.7.6
chromadb                                 0.6.3
click                                    8.1.8
cloudevents                              1.11.0
cloudpathlib                             0.20.0
cn2an                                    0.5.23
cobble                                   0.1.4
colorama                                 0.4.6
coloredlogs                              15.0.1
comm                                     0.2.2
confection                               0.1.5
conllu                                   4.5.3
contourpy                                1.3.1
courlan                                  1.3.2
cryptography                             44.0.1
csvw                                     3.5.1
curated-tokenizers                       0.0.9
curated-transformers                     0.1.1
cycler                                   0.12.1
cymem                                    2.0.11
datasets                                 3.3.2
dateparser                               1.2.1
debugpy                                  1.8.12
decorator                                5.1.1
defusedxml                               0.7.1
deprecated                               1.2.18
deprecation                              2.1.0
devtools                                 0.12.2
dill                                     0.3.8
diskcache                                5.6.3
distro                                   1.9.0
dlinfo                                   2.0.0
dnspython                                2.7.0
docopt                                   0.6.2
durationpy                               0.9
easyocr                                  1.7.2
einops                                   0.8.1
email-validator                          2.2.0
en-core-web-sm                           3.8.0
environs                                 11.2.1
espeakng-loader                          0.2.4
et-xmlfile                               2.0.0
evaluate                                 0.4.3
executing                                2.2.0
fastapi                                  0.115.8
fastapi-cli                              0.0.7
fastjsonschema                           2.21.1
ffmpeg-python                            0.2.0
filelock                                 3.17.0
filetype                                 1.2.0
fireworks-ai                             0.15.12
flair                                    0.15.1
flatbuffers                              25.2.10
fnllm                                    0.0.10
fonttools                                4.56.0
fqdn                                     1.5.1
frozenlist                               1.5.0
fsspec                                   2024.12.0
ftfy                                     6.3.1
fugashi                                  1.4.0
future                                   1.0.0
gast                                     0.6.0
gdown                                    5.2.0
gensim                                   4.3.3
google-ai-generativelanguage             0.6.15
google-api-core                          2.24.1
google-api-python-client                 2.161.0
google-auth                              2.38.0
google-auth-httplib2                     0.2.0
google-generativeai                      0.8.4
google-pasta                             0.2.0
googleapis-common-protos                 1.68.0
graphrag                                 1.2.0
graphviz                                 0.20.3
graspologic                              3.4.1
graspologic-native                       1.2.3
greenlet                                 3.1.1
grpcio                                   1.67.1
grpcio-status                            1.67.1
h11                                      0.14.0
h5py                                     3.13.0
hdbscan                                  0.8.40
html2text                                2024.2.26
htmldate                                 1.9.3
httpcore                                 1.0.7
httplib2                                 0.22.0
httptools                                0.6.4
httpx                                    0.28.1
httpx-sse                                0.4.0
httpx-ws                                 0.7.1
huggingface-hub                          0.29.1
humanfriendly                            10.0
hyppo                                    0.4.0
idna                                     3.10
imageio                                  2.37.0
importlib-metadata                       8.5.0
importlib-resources                      6.5.2
inquirerpy                               0.3.4
intervaltree                             3.1.0
ipykernel                                6.29.5
ipython                                  8.32.0
isodate                                  0.7.2
isoduration                              20.11.0
jaconv                                   0.4.0
jedi                                     0.19.2
jieba                                    0.42.1
jinja2                                   3.1.5
jiter                                    0.8.2
jmespath                                 1.0.1
joblib                                   1.4.2
json-repair                              0.30.3
json5                                    0.10.0
jsonlines                                4.0.0
jsonpatch                                1.33
jsonpointer                              3.0.0
jsonref                                  1.1.0
jsonschema                               4.23.0
jsonschema-path                          0.3.4
jsonschema-specifications                2024.10.1
jupyter-client                           8.6.3
jupyter-core                             5.7.2
jupyter-events                           0.12.0
jupyter-lsp                              2.2.5
jupyter-server                           2.15.0
jupyter-server-terminals                 0.5.3
jupyterlab                               4.3.5
jupyterlab-pygments                      0.3.0
jupyterlab-server                        2.27.3
justext                                  3.0.1
keras                                    3.8.0
kiwisolver                               1.4.8
kokoro                                   0.7.16
kubernetes                               32.0.1
lancedb                                  0.17.0
langchain                                0.3.14
langchain-anthropic                      0.3.3
langchain-aws                            0.2.13
langchain-core                           0.3.37
langchain-fireworks                      0.2.7
langchain-google-genai                   2.0.8
langchain-ollama                         0.2.2
langchain-openai                         0.3.1
langchain-text-splitters                 0.3.6
langcodes                                3.5.0
langdetect                               1.0.9
langsmith                                0.2.11
language-data                            1.3.0
language-tags                            1.2.0
lazy-loader                              0.4
lazy-object-proxy                        1.10.0
libclang                                 18.1.1
llvmlite                                 0.44.0
lmnr                                     0.4.61
loguru                                   0.7.3
lxml                                     5.3.1
lxml-html-clean                          0.4.1
maincontentextractor                     0.0.4
mako                                     1.3.9
mammoth                                  1.9.0
marisa-trie                              1.2.1
markdown                                 3.7
markdown-it-py                           3.0.0
markdownify                              0.14.1
markitdown                               0.0.1a4
markupsafe                               3.0.2
marshmallow                              3.26.1
matplotlib                               3.10.0
matplotlib-inline                        0.1.7
mdurl                                    0.1.2
misaki                                   0.7.17
mistune                                  3.1.2
ml-dtypes                                0.4.1
mlxtend                                  0.23.4
mmh3                                     5.1.0
mojimoji                                 0.0.13
monotonic                                1.6
more-itertools                           10.6.0
mpld3                                    0.5.10
mpmath                                   1.3.0
msal                                     1.31.1
msal-extensions                          1.2.0
multidict                                6.1.0
multiprocess                             0.70.16
murmurhash                               1.0.12
namex                                    0.0.8
nbclient                                 0.10.2
nbconvert                                7.16.6
nbformat                                 5.10.4
nest-asyncio                             1.6.0
networkx                                 3.4.2
ninja                                    1.11.1.3
nltk                                     3.9.1
notebook-shim                            0.2.4
num2words                                0.5.14
numba                                    0.61.0
numpy                                    1.26.4
oauthlib                                 3.2.2
olefile                                  0.47
ollama                                   0.4.7
onnx                                     1.17.0
onnxruntime                              1.20.1
onnxruntime-genai                        0.7.0rc1
openai                                   1.63.2
openai-whisper                           20240930
openapi-core                             0.19.4
openapi-schema-validator                 0.6.3
openapi-spec-validator                   0.7.1
opencv-python                            4.11.0.86
opencv-python-headless                   4.11.0.86
openpyxl                                 3.1.5
opentelemetry-api                        1.30.0
opentelemetry-exporter-otlp-proto-common 1.30.0
opentelemetry-exporter-otlp-proto-grpc   1.30.0
opentelemetry-exporter-otlp-proto-http   1.30.0
opentelemetry-instrumentation            0.51b0
opentelemetry-instrumentation-asgi       0.51b0
opentelemetry-instrumentation-fastapi    0.51b0
opentelemetry-instrumentation-langchain  0.38.7
opentelemetry-instrumentation-requests   0.51b0
opentelemetry-instrumentation-sqlalchemy 0.51b0
opentelemetry-instrumentation-threading  0.51b0
opentelemetry-instrumentation-urllib3    0.51b0
opentelemetry-proto                      1.30.0
opentelemetry-sdk                        1.30.0
opentelemetry-semantic-conventions       0.51b0
opentelemetry-semantic-conventions-ai    0.4.2
opentelemetry-util-http                  0.51b0
opt-einsum                               3.4.0
optree                                   0.14.0
ordered-set                              4.1.0
orjson                                   3.10.15
overrides                                7.7.0
packaging                                24.2
pandas                                   2.2.3
pandocfilters                            1.5.1
parse                                    1.20.2
parso                                    0.8.4
pathable                                 0.4.4
pathvalidate                             3.2.3
patsy                                    1.0.1
pdfminer-six                             20240706
peft                                     0.14.0
pexpect                                  4.9.0
pfzy                                     0.3.4
phonemizer-fork                          3.3.2
pillow                                   11.1.0
pip                                      25.0.1
platformdirs                             4.3.6
playwright                               1.50.0
portalocker                              2.10.1
posthog                                  3.14.2
pot                                      0.9.5
pptree                                   3.1
prance                                   23.6.21.0
preshed                                  3.0.9
proces                                   0.1.7
prometheus-client                        0.21.1
prompt-toolkit                           3.0.50
propcache                                0.3.0
proto-plus                               1.26.0
protobuf                                 5.29.3
psutil                                   7.0.0
psycopg                                  3.2.4
ptyprocess                               0.7.0
pure-eval                                0.2.3
puremagic                                1.28
py-cpuinfo                               9.0.0
pyarrow                                  15.0.2
pyasn1                                   0.6.1
pyasn1-modules                           0.4.1
pybars4                                  0.9.13
pyclipper                                1.3.0.post6
pycparser                                2.22
pycryptodome                             3.21.0
pydantic                                 2.10.6
pydantic-core                            2.27.2
pydantic-settings                        2.8.0
pydub                                    0.25.1
pyee                                     12.1.1
pygments                                 2.19.1
pyjwt                                    2.10.1
pylance                                  0.20.0
pymeta3                                  0.5.1
pynndescent                              0.5.13
pyparsing                                3.2.1
pypika                                   0.48.9
pypinyin                                 0.53.0
pyproject-hooks                          1.2.0
pysocks                                  1.7.1
python-bidi                              0.6.6
python-dateutil                          2.9.0.post0
python-dotenv                            1.0.1
python-gitlab                            5.6.0
python-json-logger                       3.2.1
python-multipart                         0.0.20
python-pptx                              1.0.2
python-telegram-bot                      21.10
pytorch-revgrad                          0.2.0
pytz                                     2025.1
pyyaml                                   6.0.2
pyzbar                                   0.1.9
pyzmq                                    26.2.1
rdflib                                   7.1.3
referencing                              0.36.2
regex                                    2024.11.6
requests                                 2.32.3
requests-oauthlib                        2.0.0
requests-toolbelt                        1.0.0
rfc3339-validator                        0.1.4
rfc3986                                  1.5.0
rfc3986-validator                        0.1.1
rich                                     13.9.4
rich-toolkit                             0.13.2
rpds-py                                  0.23.0
rsa                                      4.9
ruamel-yaml                              0.18.10
ruamel-yaml-clib                         0.2.12
s3transfer                               0.11.2
safetensors                              0.5.2
scikit-image                             0.25.2
scikit-learn                             1.6.1
scipy                                    1.12.0
seaborn                                  0.13.2
segments                                 2.3.0
segtok                                   1.5.11
semantic-kernel                          1.20.0
send2trash                               1.8.3
sentence-transformers                    3.4.1
sentencepiece                            0.2.0
setuptools                               75.8.0
shapely                                  2.0.7
shellingham                              1.5.4
six                                      1.17.0
smart-open                               7.1.0
sniffio                                  1.3.1
sortedcontainers                         2.4.0
soundfile                                0.13.1
soupsieve                                2.6
spacy                                    3.8.4
spacy-curated-transformers               0.3.0
spacy-legacy                             3.0.12
spacy-loggers                            1.0.5
speechrecognition                        3.14.1
sqlalchemy                               2.0.38
sqlitedict                               2.1.0
sqlmodel                                 0.0.22
srsly                                    2.5.1
stack-data                               0.6.3
starlette                                0.45.3
statsmodels                              0.14.4
sympy                                    1.13.1
tabulate                                 0.9.0
tenacity                                 9.0.0
tensorboard                              2.18.0
tensorboard-data-server                  0.7.2
tensorflow                               2.18.0
termcolor                                2.5.0
terminado                                0.18.1
tf-keras                                 2.18.0
thinc                                    8.3.4
threadpoolctl                            3.5.0
tifffile                                 2025.2.18
tiktoken                                 0.8.0
tinycss2                                 1.4.0
tld                                      0.13
tokenizers                               0.21.0
torch                                    2.6.0
torchvision                              0.21.0
tornado                                  6.4.2
tqdm                                     4.67.1
trafilatura                              2.0.0
traitlets                                5.14.3
transformer-smaller-training-vocab       0.4.0
transformers                             4.49.0
trl                                      0.15.1
typer                                    0.15.1
types-python-dateutil                    2.9.0.20241206
typing-extensions                        4.12.2
tzdata                                   2025.1
tzlocal                                  5.3
ultralytics                              8.3.78
ultralytics-thop                         2.0.14
umap-learn                               0.5.7
unidic-lite                              1.0.8
unsloth                                  2024.8
uri-template                             1.3.0
uritemplate                              4.1.1
urllib3                                  2.3.0
uvicorn                                  0.34.0
uvloop                                   0.21.0
wasabi                                   1.1.3
watchfiles                               1.0.4
wcwidth                                  0.2.13
weasel                                   0.4.1
webcolors                                24.11.1
webencodings                             0.5.1
websocket-client                         1.8.0
websockets                               15.0
werkzeug                                 3.1.3
wheel                                    0.45.1
wikipedia-api                            0.8.1
wrapt                                    1.17.2
wsproto                                  1.2.0
xlrd                                     2.0.1
xlsxwriter                               3.2.2
xxhash                                   3.5.0
yarl                                     1.18.3
youtube-transcript-api                   0.6.3
z3-solver                                4.14.0.0
zipp                                     3.21.0

Instructions I've followed here on huggingface:

# Download the model directly using the Hugging Face CLI
huggingface-cli download microsoft/Phi-4-mini-instruct-onnx --include cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/* --local-dir .

# Install the CPU package of ONNX Runtime GenAI
pip install --pre onnxruntime-genai

# Please adjust the model directory (-m) accordingly
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi3-qa.py -o phi3-qa.py
python phi3-qa.py -m cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4 -e cpu

[leestott]

Phi-4 mini and Phi-3 have different architectures so you will not be able to run Phi-3 samples on Phi-4-mini or Phi-4-multimodal

[sushraja-msft]

Update: This is potentially a bug in Ort that is being fixed here microsoft/onnxruntime-extensions#905