Hello, 

I am trying to import a trained model in PyTorch. It consists of a tokenizer and another model that is very large. However, I started with the tokenizer using the following: -

 

tokenizer = qb.Download("DropBox URL")
import torch
tokenizer_model = torch.load(tokenizer)

 

But I ended with the following error:-

ValueError                                Traceback (most recent call last)
<ipython-input-28-acfbb532fa2a> in <module>
      1 import torch
----> 2 tokenizer_model = torch.load(tokenizer)

/opt/miniconda3/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    582         pickle_load_args['encoding'] = 'utf-8'
    583 
--> 584     with _open_file_like(f, 'rb') as opened_file:
    585         if _is_zipfile(opened_file):
    586             with _open_zipfile_reader(f) as opened_zipfile:

/opt/miniconda3/lib/python3.6/site-packages/torch/serialization.py in _open_file_like(name_or_buffer, mode)
    232 def _open_file_like(name_or_buffer, mode):
    233     if _is_path(name_or_buffer):
--> 234         return _open_file(name_or_buffer, mode)
    235     else:
    236         if 'w' in mode:

/opt/miniconda3/lib/python3.6/site-packages/torch/serialization.py in __init__(self, name, mode)
    213 class _open_file(_opener):
    214     def __init__(self, name, mode):
--> 215         super(_open_file, self).__init__(open(name, mode))
    216 
    217     def __exit__(self, *args):

ValueError: embedded null byte

I found an old thread, but unfortunately the author did not share the exact solution he found.