¿Qué es un os.chdir y un mkdir específicos de subprocesos en python?

I have several threads each of which is changing working directories and occasionally creating new ones in specific working directories, copying/moving files etc in these directories. Think e.g.:

def thread1:
  while True:
    os.chdir('dir')
    os.mkdir('newdir')
    os.system('mv *.png newdir/')
    do something

def thread2:
  while True:
    os.chdir('another-dir')
    os.mkdir('another-newdir')
    os.system('mv *.png another-newdir/')
    do something

I've read that chdir, mkdir functions are not specific to threads but global. What is a way to accomplish this? I can try to use absolute paths but is that the best solution?

preguntado el 05 de mayo de 13 a las 20:05

A quick way to solve it is to never call os.chdir(). In your case it would become os.mkdir('dir/newdir'); os.system('cd dir && mv *.png newdir/'). Tenga en cuenta que el cd is inside the shell command. -

Yet Another solution --only on Linux-- is to use unshare(CLONE_FS) function, but that definitely requires writing C or using ctypes/cffi. -

3 Respuestas

The working directory is an information of the MODO DE PREPARACIÓN, hence all threads share the same working directory. You debe: work with absolute paths if you want to use multiple threads.

You can obtain absolute paths quite easily using the os.path módulo.

An other thing that you may consider is using python's standard library to do the copying instead of calling external processes.

Ver por ejemplo:

contestado el 05 de mayo de 13 a las 20:05

Probably not very useful, but there exists a solution on Linux (which does not exist on other OSes, and is not exposed to Python AFAIK): do man mkdirat Para descubrir mas. - armin rigo

@ArminRigo, mkdirat es POSIX. - almohada

@pilcrow: You're right. Also, it has been added to Python 3.3 where you can say for example os.open(..., dir_fd=fd). - armin rigo

@ArminRigo I didn't know about mkdirat, but I'd still prefer avoiding doing this and explicitly use absolute paths. - Bakuriú

The Current Working Directory at OS level usually is process specific, not per thread. (Linux: see dejar de compartir)

Python internal file operations mostly can be re-written to use a os.path.join() y evitar chdir().

When this is not possible, e.g. for unchangeable code, for RExec kind of sandbox executions or so, then you could provide a virtual os módulo, open functions etc. which do path adjustment behind the scenes.

For calling external programs, you can provide a cwd=... argument in subprocess.Popen() calls and friends.

contestado el 11 de mayo de 17 a las 14:05

Instead of using multiple threads, you could as well use multiple processes which solves the issue.

Busque os.fork().

I have backup script that work on multiple disks, and by using a process per disk, I can make sure disk utilization stays at 100% on all disks.

There are a variety of other libraries to manage process pools in similar ways to thread pools, my favorite is the competidor paquete.

Respondido 09 Feb 21, 22:02

No es la respuesta que estás buscando? Examinar otras preguntas etiquetadas or haz tu propia pregunta.