This is a testbed to play around with transformers, and get one to train (not fine tuning, just can I get the base model to work)
This is the code taken from the pytorch demo at: https://pytorch.org/tutorials/beginner/transformer_tutorial.html
I made some edits and used huggingface to load the data, since I had trouble with pytext datasets.
This is an example using the transformer I built. This is more a demo piece and not production code. I used the outline of the pytorch training and dataprep code to try and replicate results.