Reading Tokens From A File In Python 3.x

Is there some way to read in information from a file by token regardless of the formatting? For example, if I am trying to generate a ppm image from an input file and instead of 2

Solution 1:

You could always roll your own file iterator:

classfile_tokens:
    def__init__(self, file):
        self.file = file
        self.line = []
    def__iter__(self):
        return self
    defnext(self):
        whilenotlen(self.line):
            self.line = self.file.readline()
            ifnot self.line:
                raise StopIteration
            self.line = self.line.split()
        return self.line.pop(0)

Then use like a normal file:

for token in file_tokens(open(infile)):
    print('Token: ' + token)

Solution 2:

You can use chain.from_iterable, where the iter-able would be line.split() for line in fin:

>>> withopen('temp.txt', 'r') as fin:
... iter = chain.from_iterable(line.split() for line in fin)
... print(list(iter))
... 
['255', '0', '0', '0', '0', '255']