Upload tokenizer

README.md

@@ -0,0 +1,201 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+

added_tokens.json

@@ -0,0 +1,605 @@
+{
+  "<0813-124 phase II>": 32190,
+  "<090008>": 32446,
+  "<1.2.20>": 32497,
+  "<15H5D-4a>": 32428,
+  "<1692>": 32200,
+  "<174/2>": 32305,
+  "<17Nkhm-UP2>": 32525,
+  "<18EpOKYJ>": 32468,
+  "<200023>": 32153,
+  "<21A>": 32575,
+  "<24.1>": 32217,
+  "<301>": 32266,
+  "<3347689II>": 32620,
+  "<3937>": 32490,
+  "<477>": 32459,
+  "<49125>": 32498,
+  "<5D>": 32375,
+  "<640>": 32504,
+  "<670-83>": 32312,
+  "<675>": 32419,
+  "<6D370>": 32381,
+  "<757>": 32517,
+  "<78-1320>": 32391,
+  "<7A>": 32199,
+  "<80813>": 32120,
+  "<>": 32350,
+  "<A1122>": 32144,
+  "<A1>": 32401,
+  "<A2-F21>": 32388,
+  "<A212-S19-A16>": 32335,
+  "<A23BA>": 32556,
+  "<A398-S21-F17>": 32286,
+  "<ACYC.E9L>": 32546,
+  "<ANU1>": 32399,
+  "<AR>": 32170,
+  "<ARAD>": 32220,
+  "<AR_0082>": 32101,
+  "<AS9>": 32364,
+  "<ATCC 13028>": 32426,
+  "<ATCC 39140>": 32424,
+  "<ATCC 43969>": 32511,
+  "<ATCC 51329>": 32626,
+  "<ATCC BAA-895>": 32277,
+  "<AVS0177>": 32630,
+  "<Annandia>": 32126,
+  "<ArsBeeUS>": 32544,
+  "<Arsenophonus apicola>": 32500,
+  "<Arsenophonus endosymbiont of Aleurodicus dispersus>": 32475,
+  "<Arsenophonus endosymbiont of Aphis craccivora>": 32386,
+  "<Arsenophonus nasoniae>": 32509,
+  "<Arsenophonus>": 32605,
+  "<Atlantibacter hermannii>": 32242,
+  "<Atlantibacter subterranea>": 32593,
+  "<Atlantibacter>": 32239,
+  "<BDA62-3>": 32159,
+  "<BHKY>": 32321,
+  "<BO-1>": 32215,
+  "<BPEN>": 32403,
+  "<BVAF>": 32308,
+  "<BY21311>": 32290,
+  "<Bacteria>": 32613,
+  "<Blochmannia endosymbiont of Camponotus (Colobopsis) obliquus>": 32535,
+  "<Blochmannia endosymbiont of Camponotus modoc>": 32244,
+  "<Blochmannia endosymbiont of Camponotus nipponensis>": 32135,
+  "<Blochmannia endosymbiont of Colobopsis nipponica>": 32313,
+  "<Blochmannia endosymbiont of Polyrhachis (Hedomyrma) turneri>": 32582,
+  "<Blochmannia>": 32336,
+  "<Brenneria goodwinii>": 32276,
+  "<Brenneria izadpanahii>": 32284,
+  "<Brenneria nigrifluens>": 32531,
+  "<Brenneria rubrifaciens>": 32137,
+  "<Brenneria ulupoensis>": 32385,
+  "<Brenneria>": 32275,
+  "<Bruguierivoracaceae>": 32229,
+  "<Buchnera aphidicola>": 32484,
+  "<Buchnera>": 32528,
+  "<Budviciaceae>": 32304,
+  "<Buttiauxella agrestis>": 32193,
+  "<Buttiauxella ferragutiae>": 32564,
+  "<Buttiauxella>": 32518,
+  "<C-002>": 32506,
+  "<C-005>": 32256,
+  "<C-006>": 32596,
+  "<C-050>": 32634,
+  "<C-7-2>": 32577,
+  "<CAVP490>": 32488,
+  "<CB>": 32631,
+  "<CCA6>": 32410,
+  "<CCUG 66741>": 32378,
+  "<CF-458>": 32293,
+  "<CFBP 3304>": 32332,
+  "<CFCC10813>": 32296,
+  "<CFPB1430>": 32465,
+  "<CFS1934>": 32192,
+  "<CQ10>": 32281,
+  "<CS-931>": 32251,
+  "<Candidatus Arsenophonus lipoptenae>": 32325,
+  "<Candidatus Blochmannia pennsylvanicus>": 32353,
+  "<Candidatus Blochmannia vafer>": 32320,
+  "<Candidatus Doolittlea endobia>": 32303,
+  "<Candidatus Fukatsuia symbiotica>": 32435,
+  "<Candidatus Gullanella endobia>": 32125,
+  "<Candidatus Hoaglandella endobia>": 32594,
+  "<Candidatus Mikella endobia>": 32372,
+  "<Candidatus Purcelliella pentastirinorum>": 32578,
+  "<Candidatus Riesia pediculicola>": 32489,
+  "<Candidatus Tachikawaea gelatinosa>": 32362,
+  "<Candidatus Westeberhardia cardiocondylae>": 32571,
+  "<Candidatus blochmannia chromaiodes>": 32421,
+  "<Candidatus ishikawaella capsulata>": 32141,
+  "<Candidatus moranella endobia>": 32139,
+  "<Candidatus sodalis pierantonius>": 32148,
+  "<Candidatus>": 32536,
+  "<Candidatus_antoea carbekii>": 32282,
+  "<Candidatus_ukatsuia>": 32576,
+  "<Cedecea lapagei>": 32261,
+  "<Cedecea neteri>": 32379,
+  "<Cedecea>": 32208,
+  "<Cf7303>": 32597,
+  "<Chania multitudinisentens>": 32356,
+  "<Chania>": 32219,
+  "<Citrobacter amalonaticus>": 32516,
+  "<Citrobacter arsenatis>": 32456,
+  "<Citrobacter braakii>": 32233,
+  "<Citrobacter freundii>": 32310,
+  "<Citrobacter koseri>": 32591,
+  "<Citrobacter portucalensis>": 32295,
+  "<Citrobacter rodentium>": 32572,
+  "<Citrobacter sedlakii>": 32469,
+  "<Citrobacter tructae>": 32342,
+  "<Citrobacter werkmanii>": 32523,
+  "<Citrobacter>": 32584,
+  "<Cp2>": 32514,
+  "<Cronobacter condimenti>": 32603,
+  "<Cronobacter dublinensis>": 32166,
+  "<Cronobacter malonaticus>": 32463,
+  "<Cronobacter muytjensii>": 32327,
+  "<Cronobacter sakazakii>": 32173,
+  "<Cronobacter universalis>": 32464,
+  "<Cronobacter>": 32568,
+  "<DH-S01>": 32147,
+  "<DSM 101947>": 32156,
+  "<DSM 102253>": 32610,
+  "<DSM 107547>": 32280,
+  "<DSM 15199>": 32246,
+  "<DSM 16636>": 32105,
+  "<DSM 16690>": 32533,
+  "<DSM 22758>": 32413,
+  "<DSM 32899>": 32441,
+  "<DSM 4481>": 32411,
+  "<DSM 4576>": 32343,
+  "<DSM 9389>": 32112,
+  "<Dickeya aquatica>": 32330,
+  "<Dickeya chrysanthemi>": 32513,
+  "<Dickeya dadantii>": 32376,
+  "<Dickeya dianthicola>": 32422,
+  "<Dickeya fangzhongdai>": 32340,
+  "<Dickeya parazeae>": 32574,
+  "<Dickeya poaceiphila>": 32481,
+  "<Dickeya solani>": 32367,
+  "<Dickeya zeae>": 32225,
+  "<Dickeya>": 32152,
+  "<Doolittlea>": 32299,
+  "<Duffyella gerundensis>": 32527,
+  "<Duffyella>": 32237,
+  "<EBP3064>": 32167,
+  "<EN-119>": 32627,
+  "<ERMR1:05>": 32323,
+  "<Eb661>": 32473,
+  "<Ech1591>": 32117,
+  "<Ech586>": 32346,
+  "<Ech703>": 32131,
+  "<Edwardsiella anguillarum>": 32203,
+  "<Edwardsiella hoshinae>": 32108,
+  "<Edwardsiella ictaluri>": 32522,
+  "<Edwardsiella piscicida>": 32451,
+  "<Edwardsiella tarda>": 32587,
+  "<Edwardsiella>": 32185,
+  "<Enterobacter asburiae>": 32442,
+  "<Enterobacter bugandensis>": 32231,
+  "<Enterobacter chengduensis>": 32333,
+  "<Enterobacter cloacae>": 32589,
+  "<Enterobacter hormaechei>": 32339,
+  "<Enterobacter huaxiensis>": 32145,
+  "<Enterobacter ludwigii>": 32329,
+  "<Enterobacter mori>": 32492,
+  "<Enterobacter oligotrophicus>": 32583,
+  "<Enterobacter pseudoroggenkampii>": 32224,
+  "<Enterobacter roggenkampii>": 32287,
+  "<Enterobacter sichuanensis>": 32161,
+  "<Enterobacter soli>": 32440,
+  "<Enterobacter>": 32132,
+  "<Enterobacterales>": 32174,
+  "<Enterobacteriaceae endosymbiont of Macroplea mutica>": 32592,
+  "<Enterobacteriaceae endosymbiont of Plateumaris pusilla>": 32366,
+  "<Enterobacteriaceae endosymbiont of_acroplea mutica>": 32377,
+  "<Enterobacteriaceae>": 32396,
+  "<EpK1/15>": 32478,
+  "<ErCicurvipes>": 32466,
+  "<Erwinia amylovora>": 32113,
+  "<Erwinia billingiae>": 32618,
+  "<Erwinia persicina>": 32195,
+  "<Erwinia pyrifoliae>": 32434,
+  "<Erwinia rhapontici>": 32300,
+  "<Erwinia sorbitola>": 32294,
+  "<Erwinia tasmaniensis>": 32588,
+  "<Erwinia tracheiphila>": 32357,
+  "<Erwinia>": 32397,
+  "<Erwiniaceae>": 32252,
+  "<Escherichia albertii>": 32554,
+  "<Escherichia coli >": 32566,
+  "<Escherichia fergusonii>": 32600,
+  "<Escherichia marmotae>": 32461,
+  "<Escherichia>": 32415,
+  "<Et1/99>": 32326,
+  "<FDAARGOS 1447>": 32158,
+  "<FDAARGOS_1499>": 32205,
+  "<FDAARGOS_165>": 32448,
+  "<FDAARGOS_186>": 32479,
+  "<FDAARGOS_392>": 32292,
+  "<FDAARGOS_408>": 32175,
+  "<FDAARGOS_500>": 32437,
+  "<FDAARGOS_616>": 32213,
+  "<FDAARGOS_730>": 32177,
+  "<FDAARGOS_926>": 32418,
+  "<FDAARGOS_940>": 32637,
+  "<FIN>": 32314,
+  "<FN20211>": 32429,
+  "<FRB141>": 32427,
+  "<FRB97>": 32169,
+  "<FRM16>": 32408,
+  "<FY-07>": 32123,
+  "<FY158>": 32485,
+  "<G5>": 32241,
+  "<G6>": 32547,
+  "<Gammaproteobacteria>": 32311,
+  "<Gibbsiella quercinecans>": 32212,
+  "<Gibbsiella>": 32590,
+  "<Gullanella>": 32248,
+  "<H4-C11>": 32149,
+  "<HI4320>": 32143,
+  "<HS11286>": 32384,
+  "<HS1>": 32168,
+  "<HYN0051>": 32121,
+  "<Hafnia alvei>": 32482,
+  "<Hafnia paralvei>": 32298,
+  "<Hafnia>": 32234,
+  "<Hafniaceae>": 32262,
+  "<Hoaglandella>": 32405,
+  "<IFB5427>": 32230,
+  "<IP32953>": 32358,
+  "<Iran 50>": 32455,
+  "<Ishikawaella>": 32567,
+  "<J780>": 32453,
+  "<JH01>": 32259,
+  "<JK2.1>": 32390,
+  "<JZ-GX1>": 32128,
+  "<JZB2120001>": 32106,
+  "<Jejubacter calystegiae>": 32104,
+  "<Jejubacter>": 32636,
+  "<K-12 substr. MG1655>": 32501,
+  "<K61>": 32400,
+  "<KACC 18508>": 32404,
+  "<KC-Pc-HB1>": 32452,
+  "<KMM821>": 32338,
+  "<KSNA2>": 32240,
+  "<KUDC3025>": 32209,
+  "<Ka37751>": 32334,
+  "<Kalro>": 32409,
+  "<Klebsiella aerogenes>": 32425,
+  "<Klebsiella africana>": 32491,
+  "<Klebsiella electrica>": 32201,
+  "<Klebsiella huaxiensis>": 32539,
+  "<Klebsiella michiganensis>": 32608,
+  "<Klebsiella oxytoca>": 32291,
+  "<Klebsiella pasteurii>": 32433,
+  "<Klebsiella pneumoniae>": 32317,
+  "<Klebsiella quasipneumoniae>": 32499,
+  "<Klebsiella variicola>": 32188,
+  "<Klebsiella>": 32520,
+  "<Kluyvera ascorbata>": 32214,
+  "<Kluyvera intermedia>": 32503,
+  "<Kluyvera>": 32297,
+  "<Kosakonia arachidis>": 32182,
+  "<Kosakonia cowanii>": 32487,
+  "<Kosakonia oryzae>": 32467,
+  "<Kosakonia oryzendophytica>": 32543,
+  "<Kosakonia pseudosacchari>": 32545,
+  "<Kosakonia radicincitans>": 32382,
+  "<Kosakonia sacchari>": 32124,
+  "<Kosakonia>": 32138,
+  "<KqPF26>": 32196,
+  "<L6>": 32559,
+  "<LEMB11>": 32551,
+  "<LF7a>": 32265,
+  "<LH84-a>": 32632,
+  "<LJ1>": 32483,
+  "<LMG 23823>": 32345,
+  "<LMG 23826>": 32560,
+  "<LMG 24197>": 32194,
+  "<LMG 24199>": 32114,
+  "<LMG 26250>": 32436,
+  "<LMG24200>": 32604,
+  "<LST-1>": 32476,
+  "<LT-1>": 32103,
+  "<LT2>": 32111,
+  "<LTYR-11Z>": 32406,
+  "<LY-1>": 32581,
+  "<Leclercia adecarboxylata>": 32157,
+  "<Leclercia pneumoniae>": 32228,
+  "<Leclercia>": 32598,
+  "<Lelliottia steviae>": 32563,
+  "<Lelliottia>": 32172,
+  "<Leminorella richardii>": 32524,
+  "<Leminorella>": 32502,
+  "<Limnobaculum parvum>": 32540,
+  "<Limnobaculum zhutongyuii>": 32477,
+  "<Limnobaculum>": 32183,
+  "<Lonsdalea britannica>": 32617,
+  "<Lonsdalea populi>": 32537,
+  "<Lonsdalea>": 32457,
+  "<Lsch>": 32417,
+  "<ME23>": 32530,
+  "<MS2>": 32316,
+  "<MiY-A>": 32557,
+  "<Mikella>": 32373,
+  "<Mixta gaviniae>": 32184,
+  "<Mixta hanseatica>": 32460,
+  "<Mixta intestinalis>": 32180,
+  "<Mixta>": 32628,
+  "<Moellerella wisconsensis>": 32267,
+  "<Moellerella>": 32619,
+  "<Moranella>": 32486,
+  "<Morganella morganii>": 32398,
+  "<Morganella>": 32322,
+  "<Morganellaceae>": 32165,
+  "<Mpkobe>": 32580,
+  "<Musicola paradisiaca>": 32430,
+  "<Musicola>": 32176,
+  "<N-5-1>": 32245,
+  "<N2-1>": 32211,
+  "<N268-08>": 32370,
+  "<NA>": 32622,
+  "<NCPPB 569>": 32109,
+  "<NCTC 14382>": 32392,
+  "<NCTC 9529>": 32337,
+  "<NCTC11466>": 32359,
+  "<NCTC12003>": 32407,
+  "<NCTC12148>": 32179,
+  "<NCTC12151>": 32602,
+  "<NCTC12284>": 32278,
+  "<NCTC13188>": 32361,
+  "<NIBIO1392>": 32206,
+  "<OLIH>": 32223,
+  "<Ola 51>": 32130,
+  "<PA13>": 32470,
+  "<PCVAL>": 32599,
+  "<PPO 9019>": 32250,
+  "<PR-310>": 32606,
+  "<PRI-2C>": 32178,
+  "<Pantoea agglomerans>": 32462,
+  "<Pantoea alfalfae>": 32526,
+  "<Pantoea alhagi>": 32247,
+  "<Pantoea ananatis>": 32163,
+  "<Pantoea deleyi>": 32198,
+  "<Pantoea dispersa>": 32493,
+  "<Pantoea eucalypti>": 32255,
+  "<Pantoea eucrina>": 32579,
+  "<Pantoea soli>": 32363,
+  "<Pantoea stewartii>": 32227,
+  "<Pantoea vagans>": 32341,
+  "<Pantoea>": 32122,
+  "<Pectobacteriaceae>": 32629,
+  "<Pectobacterium aquaticum>": 32555,
+  "<Pectobacterium aroidearum>": 32348,
+  "<Pectobacterium atrosepticum>": 32154,
+  "<Pectobacterium brasiliense>": 32127,
+  "<Pectobacterium cacticida>": 32257,
+  "<Pectobacterium carotovorum>": 32640,
+  "<Pectobacterium colocasium>": 32302,
+  "<Pectobacterium odoriferum>": 32496,
+  "<Pectobacterium parmentieri>": 32197,
+  "<Pectobacterium parvum>": 32355,
+  "<Pectobacterium polaris>": 32100,
+  "<Pectobacterium punjabense>": 32115,
+  "<Pectobacterium quasiaquaticum>": 32447,
+  "<Pectobacterium wasabiae>": 32186,
+  "<Pectobacterium>": 32306,
+  "<Photorhabdus akhurstii>": 32134,
+  "<Photorhabdus asymbiotica>": 32315,
+  "<Photorhabdus laumondii>": 32243,
+  "<Photorhabdus thracensis>": 32191,
+  "<Photorhabdus>": 32565,
+  "<Phytobacter diazotrophicus>": 32445,
+  "<Phytobacter>": 32393,
+  "<Plesiomonas shigelloides>": 32458,
+  "<Plesiomonas>": 32601,
+  "<Pluralibacter gergoviae>": 32309,
+  "<Pluralibacter>": 32611,
+  "<Pragia fontium>": 32623,
+  "<Pragia>": 32402,
+  "<Profftia>": 32273,
+  "<Proteus hauseri>": 32609,
+  "<Proteus mirabilis>": 32274,
+  "<Proteus penneri>": 32439,
+  "<Proteus terrae>": 32351,
+  "<Proteus>": 32561,
+  "<Providencia alcalifaciens>": 32238,
+  "<Providencia hangzhouensis>": 32431,
+  "<Providencia heimbachae>": 32585,
+  "<Providencia huaxiensis>": 32171,
+  "<Providencia rettgeri>": 32616,
+  "<Providencia stuartii>": 32331,
+  "<Providencia>": 32324,
+  "<Pseudocitrobacter corydidari>": 32368,
+  "<Pseudocitrobacter>": 32129,
+  "<Pseudomonadota>": 32218,
+  "<Purcelliella>": 32552,
+  "<RB-25>": 32318,
+  "<Rahnella aceris>": 32307,
+  "<Rahnella sikkimica>": 32289,
+  "<Rahnella victoriana>": 32118,
+  "<Rahnella>": 32416,
+  "<Raoultella planticola>": 32438,
+  "<Raoultella terrigena>": 32383,
+  "<Raoultella>": 32534,
+  "<Riesia>": 32633,
+  "<S07-698>": 32260,
+  "<S178-2>": 32202,
+  "<S1>": 32515,
+  "<S2-A69>": 32449,
+  "<SCPM-O-B-7604>": 32380,
+  "<SE6-1>": 32352,
+  "<SGAir0282>": 32136,
+  "<SII>": 32235,
+  "<SK>": 32494,
+  "<SNU WT2>": 32288,
+  "<SOPE>": 32558,
+  "<SRCM103226>": 32181,
+  "<SS95>": 32638,
+  "<SWHEFF_49>": 32512,
+  "<Sakai substr. RIMD 0509952>": 32387,
+  "<Salmonella bongori>": 32395,
+  "<Salmonella enterica>": 32541,
+  "<Salmonella>": 32519,
+  "<Sample 167>": 32454,
+  "<Sb-24>": 32160,
+  "<Scandinavium goeteborgense>": 32369,
+  "<Scandinavium>": 32347,
+  "<Schneideria>": 32607,
+  "<Serratia entomophila>": 32150,
+  "<Serratia ficaria>": 32480,
+  "<Serratia fonticola>": 32625,
+  "<Serratia inhibens>": 32349,
+  "<Serratia liquefaciens>": 32521,
+  "<Serratia nematodiphila>": 32360,
+  "<Serratia plymuthica>": 32116,
+  "<Serratia proteamaculans>": 32508,
+  "<Serratia quinivorans>": 32271,
+  "<Serratia rhizosphaerae>": 32444,
+  "<Serratia rubidaea>": 32570,
+  "<Serratia surfactantfaciens>": 32187,
+  "<Serratia symbiotica>": 32573,
+  "<Serratia ureilytica>": 32507,
+  "<Serratia>": 32389,
+  "<Shigella dysenteriae>": 32344,
+  "<Shigella flexneri>": 32614,
+  "<Shigella sonnei>": 32110,
+  "<Shigella>": 32548,
+  "<Shimwellia blattae>": 32553,
+  "<Shimwellia>": 32569,
+  "<Siccibacter colletis>": 32102,
+  "<Siccibacter>": 32162,
+  "<Sodalis endosymbiont of Henestaris halophilus>": 32270,
+  "<Sodalis glossinidius>": 32538,
+  "<Sodalis praecaptivus>": 32279,
+  "<Sodalis>": 32319,
+  "<SyEd1>": 32495,
+  "<Symbiopectobacterium purcellii>": 32226,
+  "<Symbiopectobacterium>": 32365,
+  "<T6>": 32472,
+  "<TA9759>": 32423,
+  "<TBY01>": 32505,
+  "<THO-011>": 32639,
+  "<TTO1>": 32549,
+  "<Tachikawaea>": 32253,
+  "<Tatumella citrea>": 32510,
+  "<Tatumella>": 32264,
+  "<Trabulsiella odontotermitis>": 32285,
+  "<Trabulsiella>": 32222,
+  "<US>": 32164,
+  "<USDA-ARS-USMARC-60222>": 32254,
+  "<USDA>": 32189,
+  "<UwTKB>": 32207,
+  "<VKH10>": 32615,
+  "<W65>": 32471,
+  "<WCHECl-C4 = WCHECh050004>": 32586,
+  "<WCHKl090001>": 32107,
+  "<WCHPr000369>": 32119,
+  "<WPP14>": 32354,
+  "<Westeberhardia>": 32432,
+  "<Wigglesworthia glossinidia>": 32155,
+  "<Wigglesworthia>": 32621,
+  "<Winslowiella toletana>": 32394,
+  "<Winslowiella>": 32236,
+  "<XL123>": 32133,
+  "<XL95>": 32412,
+  "<Xenorhabdus budapestensis>": 32151,
+  "<Xenorhabdus doucetiae>": 32263,
+  "<Xenorhabdus griffiniae>": 32216,
+  "<Xenorhabdus hominickii>": 32142,
+  "<Xenorhabdus nematophila>": 32269,
+  "<Xenorhabdus poinarii>": 32204,
+  "<Xenorhabdus>": 32232,
+  "<YD25>": 32612,
+  "<YF8>": 32542,
+  "<YRA>": 32550,
+  "<YSD YN2>": 32624,
+  "<Y_sim_228>": 32374,
+  "<Yersinia aldovae>": 32272,
+  "<Yersinia alsatica>": 32328,
+  "<Yersinia canariae>": 32450,
+  "<Yersinia hibernica>": 32301,
+  "<Yersinia intermedia>": 32258,
+  "<Yersinia mollaretii>": 32283,
+  "<Yersinia pestis>": 32532,
+  "<Yersinia pseudotuberculosis>": 32210,
+  "<Yersinia rohdei>": 32268,
+  "<Yersinia ruckeri>": 32420,
+  "<Yersinia similis>": 32635,
+  "<Yersinia>": 32146,
+  "<Yersiniaceae>": 32443,
+  "<ZJ-FGZX1>": 32595,
+  "<ZN2>": 32474,
+  "<[Enterobacter] lignolyticus>": 32562,
+  "<[Pantoea] beijingensis>": 32249,
+  "<morsitans>": 32221,
+  "<obscurior>": 32371,
+  "<secondary endosymbiont of Ctenarytaina eucalypti>": 32414,
+  "<secondary endosymbiont of Heteropsylla cubana>": 32529,
+  "<secondary endosymbiont of Trabutina mannipara>": 32140,
+  "AAA": 32681,
+  "AAC": 32680,
+  "AAG": 32682,
+  "AAU": 32679,
+  "ACG": 32666,
+  "ACU": 32665,
+  "AGA": 32697,
+  "AGC": 32696,
+  "AGG": 32698,
+  "AGU": 32695,
+  "AUA": 32651,
+  "AUC": 32650,
+  "AUG": 32652,
+  "AUU": 32649,
+  "CAA": 32677,
+  "CAC": 32676,
+  "CAG": 32678,
+  "CAU": 32675,
+  "CCA": 32663,
+  "CCC": 32662,
+  "CCG": 32664,
+  "CCU": 32661,
+  "CGA": 32693,
+  "CGC": 32692,
+  "CGG": 32694,
+  "CGU": 32691,
+  "CUA": 32647,
+  "CUC": 32646,
+  "CUG": 32648,
+  "CUU": 32645,
+  "GAA": 32685,
+  "GAC": 32684,
+  "GAG": 32686,
+  "GAU": 32683,
+  "GCA": 32669,
+  "GCC": 32668,
+  "GCG": 32670,
+  "GCU": 32667,
+  "GGA": 32701,
+  "GGC": 32700,
+  "GGG": 32702,
+  "GGU": 32699,
+  "GUA": 32655,
+  "GUC": 32654,
+  "GUG": 32656,
+  "GUU": 32653,
+  "UAA": 32673,
+  "UAC": 32672,
+  "UAG": 32674,
+  "UAU": 32671,
+  "UCA": 32659,
+  "UCC": 32658,
+  "UCG": 32660,
+  "UCU": 32657,
+  "UGA": 32689,
+  "UGC": 32688,
+  "UGG": 32690,
+  "UGU": 32687,
+  "UUA": 32643,
+  "UUC": 32642,
+  "UUG": 32644,
+  "UUU": 32641
+}

special_tokens_map.json

@@ -0,0 +1,107 @@
+{
+  "additional_special_tokens": [
+    "<extra_id_0>",
+    "<extra_id_1>",
+    "<extra_id_2>",
+    "<extra_id_3>",
+    "<extra_id_4>",
+    "<extra_id_5>",
+    "<extra_id_6>",
+    "<extra_id_7>",
+    "<extra_id_8>",
+    "<extra_id_9>",
+    "<extra_id_10>",
+    "<extra_id_11>",
+    "<extra_id_12>",
+    "<extra_id_13>",
+    "<extra_id_14>",
+    "<extra_id_15>",
+    "<extra_id_16>",
+    "<extra_id_17>",
+    "<extra_id_18>",
+    "<extra_id_19>",
+    "<extra_id_20>",
+    "<extra_id_21>",
+    "<extra_id_22>",
+    "<extra_id_23>",
+    "<extra_id_24>",
+    "<extra_id_25>",
+    "<extra_id_26>",
+    "<extra_id_27>",
+    "<extra_id_28>",
+    "<extra_id_29>",
+    "<extra_id_30>",
+    "<extra_id_31>",
+    "<extra_id_32>",
+    "<extra_id_33>",
+    "<extra_id_34>",
+    "<extra_id_35>",
+    "<extra_id_36>",
+    "<extra_id_37>",
+    "<extra_id_38>",
+    "<extra_id_39>",
+    "<extra_id_40>",
+    "<extra_id_41>",
+    "<extra_id_42>",
+    "<extra_id_43>",
+    "<extra_id_44>",
+    "<extra_id_45>",
+    "<extra_id_46>",
+    "<extra_id_47>",
+    "<extra_id_48>",
+    "<extra_id_49>",
+    "<extra_id_50>",
+    "<extra_id_51>",
+    "<extra_id_52>",
+    "<extra_id_53>",
+    "<extra_id_54>",
+    "<extra_id_55>",
+    "<extra_id_56>",
+    "<extra_id_57>",
+    "<extra_id_58>",
+    "<extra_id_59>",
+    "<extra_id_60>",
+    "<extra_id_61>",
+    "<extra_id_62>",
+    "<extra_id_63>",
+    "<extra_id_64>",
+    "<extra_id_65>",
+    "<extra_id_66>",
+    "<extra_id_67>",
+    "<extra_id_68>",
+    "<extra_id_69>",
+    "<extra_id_70>",
+    "<extra_id_71>",
+    "<extra_id_72>",
+    "<extra_id_73>",
+    "<extra_id_74>",
+    "<extra_id_75>",
+    "<extra_id_76>",
+    "<extra_id_77>",
+    "<extra_id_78>",
+    "<extra_id_79>",
+    "<extra_id_80>",
+    "<extra_id_81>",
+    "<extra_id_82>",
+    "<extra_id_83>",
+    "<extra_id_84>",
+    "<extra_id_85>",
+    "<extra_id_86>",
+    "<extra_id_87>",
+    "<extra_id_88>",
+    "<extra_id_89>",
+    "<extra_id_90>",
+    "<extra_id_91>",
+    "<extra_id_92>",
+    "<extra_id_93>",
+    "<extra_id_94>",
+    "<extra_id_95>",
+    "<extra_id_96>",
+    "<extra_id_97>",
+    "<extra_id_98>",
+    "<extra_id_99>"
+  ],
+  "eos_token": "</s>",
+  "pad_token": "<pad>",
+  "unk_token": "<unk>"
+}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
+size 791656