๋ฐ˜์‘ํ˜•

๐Ÿ‘ป OS & Tools/Ubuntu 15

Ubuntu 22.04์— nvidia driver, cuda, cuDNN, torch ์„ค์น˜ํ•˜๊ธฐ (2024 ver)

์„œ๋ฒ„ ์„ธํŒ…์€ ํ•ญ์ƒ ํž˜๋“ค๋‹ค.. ๋ชฉ์ฐจ 0. ubuntu ๋ฒ„์ „ ํ™•์ธ1. nvidia driver ์„ค์น˜2. cuda ์„ค์น˜3. cuDNN ์„ค์น˜4. torch ์„ค์น˜  0. ubuntu ๋ฒ„์ „ ํ™•์ธ์ผ๋‹จ ์‚ฌ์šฉ์ค‘์ธ ubuntu์˜ ๋ฒ„์ „์„ ํ™•์ธํ•ด์ค€๋‹ค.lsb_release -a  1. nvidia driver ์„ค์น˜1-1. ์™ธ๋ถ€์ €์žฅ์†Œ ์ถ”๊ฐ€ nvidia driver์„ ํฌํ•จํ•˜๋Š” ์™ธ๋ถ€์ €์žฅ์†Œ(PPA)๋ฅผ ์ถ”๊ฐ€ํ•˜๊ณ  ํ•ด๋‹น ๋‚ด์šฉ์„ ํŒจํ‚ค์ง€ ๋ฆฌ์ŠคํŠธ์— ์—…๋ฐ์ดํŠธํ•ด์ค€๋‹ค.sudo add-apt-repository ppa:graphics-drivers/ppasudo apt update  1-2. ์„ค์น˜ ๊ฐ€๋Šฅํ•œ ๋“œ๋ผ์ด๋ฒ„ ๋ชฉ๋ก ํ™•์ธ ubuntu-drivers๋ฅผ ์ด์šฉํ•ด ์„ค์น˜ ๊ฐ€๋Šฅํ•œ nvidia driver ๋ชฉ๋ก์„ ํ™•์ธํ•œ๋‹ค.ubuntu-drivers devices..

/etc/profile, /etc/bash.bashrc, ~/.bashrc, ~/.bash_profile ํŒŒ์ผ์˜ ์ฐจ์ด์ 

์œ ์ €๊ฐ€ ๋กœ๊ทธ์ธํ•  ๋•Œ, ๋‹ค์Œ๊ณผ ๊ฐ™์€ ์ˆœ์„œ๋กœ bash ์„ค์ •ํŒŒ์ผ๋“ค์„ ์ฝ๊ฒŒ ๋œ๋‹ค. 1. /etc/profile ์‹คํ–‰: ์‹œ์Šคํ…œ ์ „์—ญ์— ๋Œ€ํ•œ ํ™˜๊ฒฝ์„ค์ •. ์ผ๋ฐ˜์ ์œผ๋กœ ๋‹ค์Œ์„ ์ˆ˜ํ–‰ํ•œ๋‹ค. - `/etc/bash.bashrc` ์‹คํ–‰ → ์‹œ์Šคํ…œ ์ „์ฒด์˜ non-login shell ํ™˜๊ฒฝ์„ค์ • - `/etc/profile.d` ๋””๋ ‰ํ† ๋ฆฌ ์•ˆ์˜ ์ถ”๊ฐ€ ์„ค์ • ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์‹คํ–‰. 2. ~/.bash_profile, ~/.bash_login, ~/.profile ์ค‘ ํ•˜๋‚˜ ์‹คํ–‰: ๊ฐœ์ธ ํ™˜๊ฒฝ ์„ค์ • - ์ผ๋ฐ˜์ ์œผ๋กœ `~/.bashrc`๋ฅผ ์‹คํ–‰ํ•œ๋‹ค - ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด login shell๊ณผ non-login shell ๋ชจ๋‘์—์„œ `~/.bashrc`๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋‹ค.

tar.gz ์••์ถ• ํ•ด์ œ ์‹œ not in gzip ์—๋Ÿฌ

tar.gz ํŒŒ์ผ์„ ์••์ถ•ํ•ด์ œํ•  ๋•Œ ๋ณดํ†ต ๋‹ค์Œ command๋ฅผ ์ด์šฉํ•˜๋Š”๋ฐ, tar -zxvf [filename.tar.gz] ๋‹ค์Œ๊ณผ ๊ฐ™์€ ์—๋Ÿฌ๊ฐ€ ๋œจ๋ฉด์„œ ์••์ถ•ํ•ด์ œ๊ฐ€ ์•ˆ๋  ๋•Œ๊ฐ€ ์žˆ๋‹ค. ์ด ๊ฒฝ์šฐ ํŒŒ์ผ์ด๋ฆ„์€ .tar.gz๋กœ ๋˜์–ด ์žˆ์ง€๋งŒ gzip ํฌ๋งท์œผ๋กœ ์••์ถ•๋˜์ง€ ์•Š์•˜์„ ์ˆ˜ ์žˆ๋‹ค. ๋‹ค์Œ command๋กœ ์ด๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋‹ค. file [filename.tar.gz] ํŒŒ์ผ์ด tar ํ˜•์‹์œผ๋กœ๋งŒ ์••์ถ•๋œ ๊ฒƒ์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋‹ค. ๋”ฐ๋ผ์„œ ์ด ๊ฒฝ์šฐ ๋‹ค์Œ command๋กœ ์••์ถ•์„ ํ•ด์ œํ•  ์ˆ˜ ์žˆ๋‹ค. tar -xvf [filename.tar.gz] ์ฐธ๊ณ : https://www.linuxfordevices.com/tutorials/linux/solved-gzip-stdin-not-in-gzip-format [SOLVED] gzip: s..

Python package dependency๋ฅผ ์•Œ๋ ค์ฃผ๋Š” pipdeptree

pipdeptree๋ผ๋Š” package๋ฅผ ์ด์šฉํ•ด ์„ค์น˜๋œ ํŒŒ์ด์ฌ ํŒจํ‚ค์ง€์˜ ์˜์กด์„ฑ์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋‹ค. ์„ค์น˜ ๋ฐฉ๋ฒ•์€ ์•„๋ž˜์™€ ๊ฐ™๋‹ค. pip install pipdeptree pipdeptree๋ฅผ ์‹คํ–‰ํ•˜๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์„ค์น˜๋œ Python package์˜ ์˜์กด์„ฑ์„ treeํ˜•ํƒœ๋กœ ๋ณด์—ฌ์ฃผ๊ณ , ๊ฐ€์žฅ ์œ„์—๋Š” ์ถฉ๋Œ์ด ์ผ์–ด๋‚  ์ˆ˜ ์žˆ๋Š” package์˜ ๋ชฉ๋ก์„ ๋ณด์—ฌ์ค€๋‹ค. -p option์„ ์ด์šฉํ•ด ํŠน์ • package์˜ ์˜์กด์„ฑ์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ์œผ๋ฉฐ, -r option์„ ์ด์šฉํ•˜๋ฉด ๋ฐ˜๋Œ€๋กœ ์–ด๋–ค package๊ฐ€ ํ•ด๋‹น package์— ์˜์กดํ•˜๊ณ  ์žˆ๋Š”์ง€๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋‹ค. package ์˜์กด์— ์ถฉ๋Œ์ด ์ผ์–ด๋‚˜๋ฉด python ์ฝ”๋“œ๋ฅผ ์‹คํ–‰ํ•  ๋•Œ ์›์ธ๋„ ์•Œ๋ ค์ฃผ์ง€ ์•Š๋Š” ์—๋Ÿฌ๊ฐ€ ๋œฐ ๋•Œ๊ฐ€ ๋งŽ์€๋ฐ.. ๐Ÿ˜ก ์œ ์šฉํ•˜๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์„ ๊ฒƒ ๊ฐ™๋‹ค.

apt ๊ด€๋ จ ๋ช…๋ น์–ด ์ •๋ฆฌ

apt์™€ apt-get์˜ ์ฐจ์ด ์ถœ์ฒ˜: https://ksbgenius.github.io/linux/2021/01/13/apt-apt-get-difference.html ์š”์•ฝํ•˜์ž๋ฉด apt-get์€ ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ์˜ต์…˜์˜ ์ˆ˜๊ฐ€ ๋งŽ๊ณ , apt๋Š” ๊ทธ ์ค‘ ๋งŽ์ด ์‚ฌ์šฉํ•˜๋Š” ์˜ต์…˜๋“ค๋งŒ ์ถ”์ถœํ•ด ๋ณด๋‹ค ํŽธ๋ฆฌํ•˜๊ณ  ๊ฐ„๋‹จํ•˜๊ฒŒ ๋งŒ๋“  ๋ฒ„์ „์ธ ๋“ฏ ํ•˜๋‹ค. ์„ค์น˜๋œ ํŒจํ‚ค์ง€ ๋ชฉ๋ก ํ™•์ธ ์ถœ์ฒ˜: ์ œํƒ€์œ„ํ‚ค dpkg -l apt --installed list apt๋ณด๋‹ค๋Š” dpkg๊ฐ€ ๊ฐ€๋…์„ฑ์ด ์ข‹์€ ๊ฒƒ ๊ฐ™๋‹ค. ํŒจํ‚ค์ง€ ์‚ญ์ œ ๊ด€๋ จ ๋ช…๋ น์–ด (remove, purge, autoremove) ์ถœ์ฒ˜: https://codechacha.com/ko/linux-apt-purge-vs-remove/ #remove sudo apt-get remove (package) sud..

CUDA, driver, PyTorch + Tensorflow ํ˜ธํ™˜๋˜๋Š” version ์ฐพ๊ณ  ์„ค์น˜(์—…๊ทธ๋ ˆ์ด๋“œ, ๋‹ค์šด๊ทธ๋ ˆ์ด๋“œ)ํ•˜๊ธฐ

CUDA๋ฅผ ์„ค์น˜ ์‹œ์— driver version์— ๋งž๋Š” version์„ ์„ค์น˜ํ•ด์•ผ ํ•˜๊ณ , CUDA version์— ๋งž๋Š” version์˜ PyTorch๋ฅผ ์„ค์น˜ํ•ด์•ผ ํ•œ๋‹ค. ์ฐธ๊ณ : Driver & Cuda & PyTorch version ํ™•์ธ Python์œผ๋กœ PyTorch, Python, CUDA, cudnn ๋ฒ„์ „ ํ™•์ธ PyTorch ๋ฒ„์ „ ํ™•์ธ import torch print(torch.__version__) Python ๋ฒ„์ „ ํ™•์ธ import sys print(sys.version) Python ๊ฒฝ๋กœ ํ™•์ธ import sys print(sys.executable) CUDA & Driver ๋ฒ„์ „ ํ™•์ธ 1. nvidia-smi - Drive.. bo-10000.tistory.com 1. Nvidia ๋“œ๋ผ์ด๋ฒ„ ์„ค..

nvidia-smi์™€ nvcc๋กœ ๋ณธ CUDA version์ด ๋‹ค๋ฅผ ๋•Œ

CUDA version์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐฉ๋ฒ•์€ ๋‘ ๊ฐ€์ง€๊ฐ€ ์žˆ๋‹ค. ํ•˜๋‚˜๋Š” nvidia-smi๋ฅผ ์ด์šฉํ•˜๋Š” ๊ฒƒ์ด๊ณ , ํ•˜๋‚˜๋Š” nvcc๋ฅผ ์ด์šฉํ•˜๋Š” ๊ฒƒ์ด๋‹ค. ๋‘˜์˜ version์ด ๋‹ฌ๋ผ์„œ ์ฐพ์•„๋ณด๋‹ˆ, ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋‹ต์„ ์–ป์„ ์ˆ˜ ์žˆ์—ˆ๋‹ค. ์ถœ์ฒ˜ : https://stackoverflow.com/questions/53422407/different-cuda-versions-shown-by-nvcc-and-nvidia-smi CUDA๋Š” 2๊ฐœ์˜ API๋ฅผ ๊ฐ€์ง€๊ณ  ์žˆ๋Š”๋ฐ, ํ•˜๋‚˜๋Š” runtime API์ด๊ณ  ๋‹ค๋ฅธ ํ•˜๋‚˜๋Š” driver API์ด๋‹ค. ๊ฐ API๊ฐ€ ๊ฐ์ž์˜ version์„ ๊ฐ€์ง€๊ณ  ์žˆ๋‹ค. driver API๋Š” GPU driver installer์— ์˜ํ•ด ์„ค์น˜๋˜๊ณ , runtime API๋Š” CUDA toolkit installer์—..

์›๊ฒฉ ์„œ๋ฒ„์— ํŒŒ์ผ ์ „์†ก(scp)

์ฐธ๊ณ  : https://harryp.tistory.com/168 usage: scp [-12346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 ... [[user@]host2:]file2 cp ๋ช…๋ น์–ด์™€ ์œ ์‚ฌ. ํŒŒ์ผ๊ฒฝ๋กœ ์•ž์— ์ถœ๋ฐœ์ง€/๋„์ฐฉ์ง€ ์ •๋ณด๋งŒ ๋ถ™์Œ. ์ถœ๋ฐœ์ง€/๋„์ฐฉ์ง€๊ฐ€ ๋กœ์ปฌ์ผ์‹œ ์ƒ๋žต. 1) ๋กœ์ปฌ-->์„œ๋ฒ„ scp [ํŒŒ์ผ๊ฒฝ๋กœ] [๋„์ฐฉ์ง€์ •๋ณด(id@ip)]:[ํŒŒ์ผ๊ฒฝ๋กœ] 2) ์„œ๋ฒ„-->๋กœ์ปฌ scp [์ถœ๋ฐœ์ง€์ •๋ณด(id@ip)]:[ํŒŒ์ผ๊ฒฝ๋กœ] [ํŒŒ์ผ๊ฒฝ๋กœ]

์„œ๋ฒ„์— jupyter notebook ์›๊ฒฉ ์—ฐ๊ฒฐ

์ฐธ๊ณ  1) https://hiseon.me/python/jupyter-notebook-install/ 2) https://blog.djjproject.com/m/487 3) https://eungbean.github.io/2019/03/22/jupyter-ssh/ 4) https://www.blopig.com/blog/2018/03/running-jupyter-notebook-on-a-remote-server-via-ssh/ 1)~3) ๋”ฐ๋ผํ•˜๋‹ค๊ฐ€ ๋ธŒ๋ผ์šฐ์ €์— localhost:9000์œผ๋กœ ์—ด๋ฉด ์•ˆ์—ด๋ ค์„œ 4)๋ฅผ ์ฐธ๊ณ . 1. (local)ssh ์ด์šฉํ•˜์—ฌ ์—ฐ๊ฒฐ ssh [id]@[server] 2. (server)๊ฐ€์ƒํ™˜๊ฒฝ ์ƒ์„ฑ conda create -n [name] 3. (server)๊ฐ€์ƒํ™˜๊ฒฝ ์ง„์ž… source..

๋ฐ˜์‘ํ˜•