Tags: acts, call, command, differently, directory, endings, ensure, file, line, program, programming, python, subprocess, thatthereare, traverse, tree, unix-style

subprocess call acts differently than command line call?

On Programmer » Python

7,396 words with 2 Comments; publish: Fri, 04 Jan 2008 16:37:00 GMT; (200203.13, « »)

Hello,

I need a program that will traverse a directory tree to ensure that

there

are unix-style line endings on every file in that tree that is a text

file.

To tell text files from others I want to use the unix "file" command

(Python's "mimetypes" is not so good for me). But I am stuck on

something about getting that output, and I'd greatly appreciate any

pointers.

Both the command line "file" and the python libmagic binding give the

same behavior, but I'll illustrate with "file". It sometimes acts

differently when run from the command line than when run using

the subprocess module (with the same user). For example, it

sometimes

gives output when run from the command line but no output when run as

a subprocess.

Below is a short program to demo. (I use this on a test file tree

that is at ftp://joshua.smcvt.edu/pub/hefferon/a.zip if anyone

is interested.)

..................................

import subprocess

import glob

for fn in glob.glob('a*/*'):

cmd=['/usr/bin/file',fn]

cmdStr=" ".join(cmd)

try:

p=subprocess.Popen(cmdStr,shell=True,stdin=subproc ess.PIPE,stdout=subprocess.PIPE,stderr=subprocess. PIPE,close_fds=True)

(child_stdin,

child_stdout,

child_stderr)=(p.stdin,p.stdout,p.stderr)

stdoutContent=child_stdout.read()

except Exception, err:

mesg=u"unable to execute %s" % (repr(cmdStr),)

raise StandardError, mesg+": "+str(err)

if (p.returncode):

mesg=u"trouble executing %s" % (repr(cmdStr),)

raise StandardError, mesg+": "+repr(p.returncode)

print "result: ",stdoutContent

print "done"

..................................

I've put a transcript of what happens at the bottom of this message.

One file (in the test tree it is "eqchange.txt") gives no output from

the above program, but does give an output when I use "file" at the

command line.

Specifying "-m/usr/share/file/magic" in the "file" call doesn't

change

that the command line and subprocess calls act differently, so it is

not just a question of different environments causing the system to

use different "magic" files. Changing the PIPE's to files, then

closing

and reopening them also does not matter, I believe.

In short I expected subprocess to just mimic my typing it in. Is

there some reason "file" doesn't act this way, and is there some

way to make it do so?

I have Python 2.4.4 running on Ubuntu. Thank you for any suggestions,

Jim

--transcript (edited to shorten)----

$ python test.py

result: acrotex/readme.txt: ASCII English text, with CRLF line

terminators

result: acrotex/eq2db.ins: ASCII English text, with CRLF line

terminators

result: acrotex/eqchange.txt:

result: acrotex/exerquiz.dtx: ISO-8859 English text, with CRLF line

terminators

result: acrotex/doc: directory

done

$ file acrotex/eqchange.txt

acrotex/eqchange.txt: ISO-8859 English text, with CRLF line

terminators

$ file acrotex/eqchange.txt 1test.out

$ cat test.out

acrotex/eqchange.txt: ISO-8859 English text, with CRLF line

terminators

$ file acrotex/eqchange.txt 2test.out

acrotex/eqchange.txt: ISO-8859 English text, with CRLF line

terminators

$ cat test.out

$

All Comments

Leave a comment...

  • 2 Comments
    • Jim <jim.hefferon.python.todaysummary.com.gmail.comwrites:

      Quote:
      === Original Words ===

      --transcript (edited to shorten)----

      $ python test.py

      result: acrotex/readme.txt: ASCII English text, with CRLF line

      terminators

      >

      result: acrotex/eq2db.ins: ASCII English text, with CRLF line

      terminators

      >

      result: acrotex/eqchange.txt:

      >

      result: acrotex/exerquiz.dtx: ISO-8859 English text, with CRLF line

      terminators

      result: acrotex/doc: directory

      >

      done

      >

      $ file acrotex/eqchange.txt

      acrotex/eqchange.txt: ISO-8859 English text, with CRLF line

      That's interesting. I've checked this eqchange.txt file from

      your a.zip on my machine and the result is:

      $ file acrotex/readme.txt

      acrotex/readme.txt: ASCII English text, with CRLF line terminators

      $ file acrotex/eqchange.txt

      acrotex/eqchange.txt:

      $ file acrotex/exerquiz.dtx

      acrotex/exerquiz.dtx: ISO-8859 English text, with CRLF line terminators

      $ file -v

      file-4.17

      magic file from /etc/magic:/usr/share/file/magic

      $ uname -orvm

      2.6.18-4-k7 #1 SMP Wed May 9 23:42:01 UTC 2007 i686 GNU/Linux

      That's really strange. Have you got only *one* version of ``file``

      program on your machine?

      --

      HTH,

      Rob

      #1; Fri, 04 Jan 2008 16:39:00 GMT
    • On Jun 7, 2:30 pm, Rob Wolfe <r....python.todaysummary.com.smsnet.plwrote:
      Quote:
      === Original Words ===

      That's really strange. Have you got only *one* version of ``file``

      program on your machine?

      Thank you, Rob. I thought I had that kind of thing covered in

      specifying the -m but obviously there was something I wasn't getting,

      and you found it.

      Hope I can return the favor sometime,

      Jim

      #2; Fri, 04 Jan 2008 16:40:00 GMT