Base Transformer Work

This is a testbed to play around with transformers, and get one to train (not fine tuning, just can I get the base model to work)

pt_demo

This is the code taken from the pytorch demo at: https://pytorch.org/tutorials/beginner/transformer_tutorial.html

I made some edits and used huggingface to load the data, since I had trouble with pytext datasets.

trans_example

This is an example using the transformer I built. This is more a demo piece and not production code. I used the outline of the pytorch training and dataprep code to try and replicate results.