Spleeter 등을 Windows에서 사용할 때 multiprocessing 관련 무한루프 오류가 생기면

Spleeter를 사용하려고 시도하였는데 아래같은 이상한 무한루프 오류를 출력하며 되질 않더라

              File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 225, in prepare
    prepare(preparation_data)  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 225, in prepare
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 225, in prepare
exitcode = _main(fd)
exitcode = _main(fd)prepare(preparation_data)
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 225, in prepare
          File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 114, in _main
    _fixup_main_from_path(data['init_main_from_path'])
_fixup_main_from_path(data['init_main_from_path'])_fixup_main_from_path(data['init_main_from_path'])_fixup_main_from_path(data['init_main_from_path'])

    _fixup_main_from_path(data['init_main_from_path'])  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path



  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 114, in _main
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 225, in prepare
prepare(preparation_data)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 225, in prepare
run_name="__mp_main__")    
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
_fixup_main_from_path(data['init_main_from_path'])
run_name="__mp_main__")  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path
prepare(preparation_data)
run_name="__mp_main__")


_fixup_main_from_path(data['init_main_from_path'])  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path
run_name="__mp_main__")  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 225, in prepare

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path
    
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
                    run_name="__mp_main__")  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path
_fixup_main_from_path(data['init_main_from_path'])
pkg_name=pkg_name, script_name=fname)
pkg_name=pkg_name, script_name=fname)  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code
pkg_name=pkg_name, script_name=fname)
pkg_name=pkg_name, script_name=fname)

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="__mp_main__")  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code

      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code

mod_name, mod_spec, pkg_name, script_name)  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code
pkg_name=pkg_name, script_name=fname)run_name="__mp_main__")
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 263, in run_path


    mod_name, mod_spec, pkg_name, script_name)              File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code
    
mod_name, mod_spec, pkg_name, script_name)
pkg_name=pkg_name, script_name=fname)pkg_name=pkg_name, script_name=fname)mod_name, mod_spec, pkg_name, script_name)      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code
pkg_name=pkg_name, script_name=fname)      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code



exec(code, run_globals)
mod_name, mod_spec, pkg_name, script_name)      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code

  File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 96, in _run_module_code
exec(code, run_globals)exec(code, run_globals)            exec(code, run_globals)    
mod_name, mod_spec, pkg_name, script_name)  File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>
mod_name, mod_spec, pkg_name, script_name)

separator = Separator('spleeter:2stems')


  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code
      File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>

mod_name, mod_spec, pkg_name, script_name)  File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code
separator = Separator('spleeter:2stems')  File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__

    exec(code, run_globals)        exec(code, run_globals)    exec(code, run_globals)

separator = Separator('spleeter:2stems')
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code

  File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>
  File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>

  File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__
self._pool = Pool()
  File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>
          File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__
          File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool
exec(code, run_globals)separator = Separator('spleeter:2stems')    self._pool = Pool()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool

  File "c:\Users\tkdrn\Desktop\sample3.py", line 7, in <module>
separator = Separator('spleeter:2stems')separator = Separator('spleeter:2stems')    separator = Separator('spleeter:2stems')
  File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__

  File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__

    separator = Separator('spleeter:2stems')            self._pool = Pool()      File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__


self._pool = Pool()  File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__
context=self.get_context())
self._pool = Pool()  File "C:\Users\tkdrn\AppData\Roaming\Python\Python37\site-packages\spleeter\separator.py", line 104, in __init__
context=self.get_context())

      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool


  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__
self._pool = Pool()self._pool = Pool()  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__
    context=self.get_context())

    self._pool = Pool()
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 119, in Pool

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__
    context=self.get_context())  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__
        context=self.get_context())        self._repopulate_pool()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__
    self._repopulate_pool()
self._repopulate_pool()  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool

      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool


context=self.get_context())    self._repopulate_pool()self._repopulate_pool()      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__
    w.start()
self._repopulate_pool()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool
w.start()      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start


self._repopulate_pool()  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 176, in __init__

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool

w.start()        w.start()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool
w.start()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start
    
    self._repopulate_pool()            self._popen = self._Popen(self)w.start()  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start
w.start()
self._popen = self._Popen(self)
self._popen = self._Popen(self)


  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\pool.py", line 241, in _repopulate_pool
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen

      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen
self._popen = self._Popen(self)        
w.start()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\process.py", line 112, in start

return Popen(process_obj)  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen
            return Popen(process_obj)        self._popen = self._Popen(self)
self._popen = self._Popen(self)


self._popen = self._Popen(self)return Popen(process_obj)      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__


return Popen(process_obj)  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
    prep_data = spawn.get_preparation_data(process_obj._name)

      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\context.py", line 322, in _Popen
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
return Popen(process_obj)    prep_data = spawn.get_preparation_data(process_obj._name)_check_not_importing_main()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    prep_data = spawn.get_preparation_data(process_obj._name)


                  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
return Popen(process_obj)      File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
is not going to be frozen to produce an executable.''')prep_data = spawn.get_preparation_data(process_obj._name)
_check_not_importing_main()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
_check_not_importing_main()
_check_not_importing_main()  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__

_check_not_importing_main()

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    RuntimeError  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    _check_not_importing_main():           File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')

        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.prep_data = spawn.get_preparation_data(process_obj._name)is not going to be frozen to produce an executable.''')    
RuntimeError


is not going to be frozen to produce an executable.''')  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
: _check_not_importing_main()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.is not going to be frozen to produce an executable.''')

에러 메세지를 읽어보는데 이게 spleeter나 tensorflow에 의존적인게 아니라 뭔가 multiprocessing이라는 거와 관련된 문제인거 같아서 더 찾아보았는데, 여러 글을 찾다보니 딱 spleeter에 맞는 아래 글이 있었다.

 

github.com/deezer/spleeter/issues/104

[Bug] Spleeter seems to be in an endless loop while separating audio · Issue #104 · deezer/spleeter

결론적으로, windows에서는 spleeter에서 사용하는 multiprocessing 모듈이 제대로 동작하지 않는댄다,,, 어이가 없다

그래서 아래와 같이 보호하면 된다더라

from spleeter.separator import Separator

if __name__ == '__main__':
    separator = Separator("spleeter:2stems")
    separator.separate_to_file("maze.wav", "")

 

뭔가 다중으로 호출되는 것을 막는 구조 같은데, 해봤더니 아주 깔끔하게 된다. (Separtor 생성자는 단독으로 사용되었을 때는 오류가 나지 않았는데, 아래 문을 호출할 때는 생성하는 문까지 저 블록 안에 넣어야 하는 것을 유의해야 한다) tensorflow gpu 관련해서 워닝메세지가 뜨긴 하는데, 결론적으로 파일 두개 잘 생성되면서 분리도 잘 되는 것을 확인했다. 찾아보니 이게 pytorch 등 저 multiprocessing을 사용하는 다양한 모듈에서 예전부터 일어나는 문제 같은데, 이런건 좀 업데이트 빨리빨리 해주면 좋겠다. 더러워서 맥 사던가 해야지