[Courses] [python] No lesson -- open discussion

[Andreas Thienemann]

On 08/21/2011 10:51 PM, Leslie wrote:

> 	First, thanks for this excellent course!
> 	I do have a question about file I/O.
> 	I had the objective of opening a file containing several lines, and
> adding lines to it.  How I solved this is probably awkward. Is there a
> better procedure or any general comments you have about working with
> files?
> I start with myfile.txt which has 2 lines:
>
> This is line 1 of my file.
> This is line 2 of my file.
>
> Then, interactively:
>>>> import os, subprocess, sys
>>>> mylist = ["This is line 3 of my file.\n", "This is line 4 of my
> file.\n"]
>>>> f = open("myfile.txt", "a")
>>>> for i in mylist:
> ...     f.write(i)
> ...
>>>> f.close()

This is actually pretty straightforward. One suggestion from me would be 
to not define the \n newline marker in the myline list but to just add 
it with the write() function. So while staying close to your code:

 >>> mylist = ["This is line 3 of my file.", "This is line 4 of my file."]
 >>> f = open("myfile.txt", "a")
 >>> for i in mylist:
...     f.write(i + "\n")
...
 >>> f.close()

This makes sure that each string will be ending up on it's own line and 
you can't make the mistake of accidentally forgetting the "\n".


Just for kicks, you can even do it without the loop by using the join() 
function which joins a string with a separator. If you chose \n as the 
separator, you get exactly what you want.

 >>> mylist = ["This is line 3 of my file.", "This is line 4 of my file."]
 >>> f = open("myfile.txt", "a")
 >>> f.write("\n".join(mylist))
 >>> f.close()



>>>> x = subprocess.Popen(["cat","myfile.txt"], stdout=subprocess.PIPE)
>>>> for line in x.stdout:
> ...     print line
> ...
> I get what I'm expecting (I closed the spaces to make this shorter),
> namely:
>
> This is line 1 of my file.
> This is line 2 of my file.
> This is line 3 of my file.
> This is line 4 of my file.
>
> Although it works, I am not sure this is the best way to add lines from
> a list.  And if I wanted to append one file to another, is there a
> straightforward way?

The adding of lines is totally fine.

But the reading part could be improved. The execution of cat on the 
shell just to read a file is needlessly complicated. After all, python 
gives you all the parts you need to read a file. No need to fall back 
onto the shell.

There are a few ways to handle that:

Open the file for reading first:
 >>> f = open("myfile.txt", "r")
 >>>

Read everything into a single string:
 >>> f.read()
'This is line 1 of my file.\nThis is line 2 of my file.\nThis is line 3 
of my file.\nThis is line 4 of my file.\n'
 >>>

You are now at the end of the file. If you read again, you won't get 
anything:
 >>> f.read()
''
 >>>

To read from the beginning again, you have to "rewind" your file to the 
beginning. The command is called seek() and takes a position in bytes. 0 
is always the beginning of the file:
 >>> f.seek(0)
 >>>

Read your file, line by line. This will read the first line and then stop.
 >>> print f.readline()
This is line 1 of my file.

 >>>
Did you notice the linebreak \n? It is read and printed to the terminal, 
resulting in a visible linebreak.

Let's use readline() a few more times and see what we get.
 >>> print f.readline()
This is line 2 of my file.

 >>> print f.readline()
This is line 3 of my file.

 >>> print f.readline()
This is line 4 of my file.

 >>> print f.readline()

 >>>

You are now at the end of the file. By putting this into a loop, you've 
already implemented the functionality to read a file and print it to the 
screen.

Or us the readlines() function which reads all lines of a file into a 
list. This is handy for nearly all files. But be careful with huge files 
as this means you'd be loading it all into memory:
 >>> f.seek(0)
 >>> f.readlines()
['This is line 1 of my file.\n', 'This is line 2 of my file.\n', 'This 
is line 3 of my file.\n', 'This is line 4 of my file.\n']
 >>>

You can use that in a loop as well and print the line with a bit of 
decoration, e.g. the filename:
 >>> f.seek(0)
 >>> for l in f.readlines():
...     print "myfile.txt:", l[:-1]
...
myfile.txt: This is line 1 of my file.
myfile.txt: This is line 2 of my file.
myfile.txt: This is line 3 of my file.
myfile.txt: This is line 4 of my file.
 >>>

After all is done, just close the file:
 >>> f.close()
 >>>

These are all more pythonesque ways of reading files.

cheers,
  andreas