๋ฐ˜์‘ํ˜•

๐ŸŒŒ Deep Learning 50

[Overview] R-CNN ๊ณ„์—ด Object Detection ์ •๋ฆฌ (Two-stage detector)

์ˆœ์„œ: 1. R-CNN (2014) 2. SPP-Net (2015) 3. Fast R-CNN (2015) 4. Faster R-CNN (2016) 5. FPN (2017) (์ถ”๊ฐ€์˜ˆ์ •) ์ฐธ๊ณ : yeomko.tistory.com/category/%EA%B0%88%EC%95%84%EB%A8%B9%EB%8A%94%20Object%20Detection?page=1 1. R-CNN (2014) Girshick, Ross, et al. "Rich feature hierarchies for accurate object detection and semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recogniti..

[PyTorch Implementation] 3D Segmentation model - VoxResNet, Attention U-Net, V-Net

PyTorch๋กœ ๊ตฌํ˜„ํ•œ 3D Segmentation ๋ชจ๋ธ 3๊ฐ€์ง€์ž…๋‹ˆ๋‹ค. github.com/bo-10000/pytorch_3d_segmentation bo-10000/pytorch_3d_segmentation PyTorch implementation of VoxResNet, Attention U-Net and V-Net - bo-10000/pytorch_3d_segmentation github.com VoxResNet residual learning์„ ์ ์šฉํ•œ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. residual module(b)์„ ์—ฌ๋Ÿฌ ๊ฒน ์Œ“์€ ๊ตฌ์กฐ๋ฅผ ๊ฐ€์ง€๊ณ  ์žˆ๊ณ , ์ค‘๊ฐ„์ค‘๊ฐ„ auxiliary classifier์„ ๋ฐฐ์น˜ํ•˜์—ฌ ๊ทธ ๊ฒฐ๊ณผ๋ฅผ ํ•ฉ์ณค์Šต๋‹ˆ๋‹ค. MICCAI MRIBrainS challenge์—์„œ 1์œ„๋ฅผ ์ฐจ์ง€ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋…ผ๋ฌธ ๋งํฌ ..

[Dataset] PASCAL VOC 2012 Segmentation ๋ฐ์ดํ„ฐ์…‹ ๋‹ค์šด๋กœ๋“œ ๋ฐ ์‚ฌ์šฉ๋ฒ•

PASCAL VOC 2012 Official Web: host.robots.ox.ac.uk/pascal/VOC/voc2012/ ๋ฐ์ดํ„ฐ์…‹ ๋‹ค์šด๋กœ๋“œ: host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit ๋ฐ์ดํ„ฐ์…‹ Docs: host.robots.ox.ac.uk/pascal/VOC/voc2012/devkit_doc.pdf PASCAL VOC 2012๋Š” ๋‹ค์Œ์˜ 5๊ฐ€์ง€ ์˜์ƒ ์ธ์‹ task๋ฅผ ์œ„ํ•ด ๊ณต๊ฐœ๋œ ๋ฐ์ดํ„ฐ์…‹์ž…๋‹ˆ๋‹ค. Classification Detection Segmentation (Semantic, Instance) Action Classification Large Scale Recognition ๋ณดํ†ต Detection ์šฉ์œผ๋กœ ๋งŽ์ด ์‚ฌ์šฉ๋˜๋Š” ๋“ฏ ํ•œ๋ฐ, Segmentation์„..

๋”ฅ๋Ÿฌ๋‹ SOTA ๋…ผ๋ฌธ ์•„์นด์ด๋ธŒ ์‚ฌ์ดํŠธ Papers with Code

paperswithcode.com/Papers with Code - About Papers With CodePapers With Code highlights trending Machine Learning research and the code to implement it.paperswithcode.com ๋‹ค์–‘ํ•œ ๋”ฅ๋Ÿฌ๋‹ ๋ถ„์•ผ์˜ SOTA ๋ชจ๋ธ์„ ํ•œ๋ˆˆ์— ๋ณผ ์ˆ˜ ์žˆ๋Š” ์•„์ฃผ ์œ ์šฉํ•œ ์‚ฌ์ดํŠธ Papers with Code์ž…๋‹ˆ๋‹ค. ์ƒ๋‹จ์˜ ๋ฉ”๋‰ด์—์„œ Browse-State-of-the-Art ํƒญ์„ ๋“ค์–ด๊ฐ€๋ฉด ์นดํ…Œ๊ณ ๋ฆฌ๋ฅผ ์„ ํƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. Vision, NLP, Medical ๋“ฑ์˜ ๋Œ€๋ถ„๋ฅ˜ ์•ˆ์— Semantic Segmentation, Image Classification ๋“ฑ์˜ ์†Œ๋ถ„๋ฅ˜๋กœ ์นดํ…Œ๊ณ ๋ฆฌ๊ฐ€ ๋‚˜๋ˆ ์ ธ ์žˆ๋Š”๋ฐ์š”, ํ˜„์žฌ ๋‚˜๋ˆ„..

[๋”ฅ๋Ÿฌ๋‹ ๋…ผ๋ฌธ๋ฆฌ๋ทฐ] Deep Double Descent: Where Bigger Models and More Data Hurt (ICLR 2020)

ICLR2020์— ๋ฐœํ‘œ๋œ ๋…ผ๋ฌธ์ธ Deep Double Descent: Where Bigger Models and More Data Hurt ๋ฅผ ๋ฐ”ํƒ•์œผ๋กœ ์ •๋ฆฌํ•œ ๊ธ€์ž…๋‹ˆ๋‹ค. ๋‹ค์–‘ํ•œ Deep Learning task์—์„œ ๋ฐœ๊ฒฌ๋˜๋Š” Double-descent๋ผ๋Š” ํ˜„์ƒ์„ Model complexity ๊ด€์ ์—์„œ ํ•ด์„ํ•˜๊ณ , ์–ด๋–ค ๊ฒฝ์šฐ์—์„œ๋Š” Model complexity๋‚˜ Train epoch๋ฅผ ์ฆ๊ฐ€์‹œํ‚ค๋Š” ๊ฒƒ์ด ์„ฑ๋Šฅ์„ ํ•˜๋ฝ์‹œํ‚ฌ ์ˆ˜๋„ ์žˆ๋‹ค๊ณ  ์ฃผ์žฅํ•ฉ๋‹ˆ๋‹ค. Classical Statistics vs. Modern Neural Networks 1) Classical Statistics: Bias-variance trade-off์— ๋”ฐ๋ฅด๋ฉด, Model complexity๊ฐ€ ์ผ์ • ์ˆ˜์ค€ ์ด์ƒ ์ปค์ง€๋ฉด Overfitting์ด ๋ฐœ์ƒํ•ด ์˜ค..

[๋”ฅ๋Ÿฌ๋‹ ๋…ผ๋ฌธ๋ฆฌ๋ทฐ] Class-Balanced Loss Based on Effective Number of Samples (CVPR 2019)

CVPR 2019์— ๋ฐœํ‘œ๋œ ๋…ผ๋ฌธ์ธ Class-Balanced Loss Based on Effective Number of Samples ๋ฅผ ์ •๋ฆฌํ•œ ๊ธ€์ž…๋‹ˆ๋‹ค. ๋ฐ์ดํ„ฐ์…‹์˜ Class Imbalance๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด ์ƒˆ๋กœ์šด Loss Design๋ฅผ ์ œ์•ˆํ•˜๋Š” ๋…ผ๋ฌธ์ž…๋‹ˆ๋‹ค. Long Tailed Dataset ์œ„ ๊ทธ๋ฆผ์€ ๊ฐ Class์— ์†ํ•˜๋Š” Sample์˜ ๊ฐฏ์ˆ˜๋ฅผ ๋‚˜ํƒ€๋‚ธ ๊ทธ๋ž˜ํ”„์ธ๋ฐ์š”, ์ผ๋ถ€ ๋ช‡ ๊ฐœ์˜ Class์—๋งŒ Sample๋“ค์ด ๋ชฐ๋ ค ์žˆ๊ณ , ๋Œ€๋ถ€๋ถ„์˜ Class์—๋Š” ๋งค์šฐ ์ ์€ ์ˆ˜์˜ Sample์ด ์žˆ๋Š” ๋ฐ์ดํ„ฐ์…‹์„ Long Tailed Dataset์ด๋ผ๊ณ  ํ•ฉ๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ๋ฐ์ดํ„ฐ์…‹์„ ๊ฐ€์ง€๊ณ  ํ•™์Šตํ•œ ๋ชจ๋ธ์€ ์ผ๋ฐ˜์ ์œผ๋กœ ์„ฑ๋Šฅ์ด ์ž˜ ๋‚˜์˜ค์ง€ ์•Š๋Š”๋ฐ, Large-scale, real-world ๋ฐ์ดํ„ฐ์…‹๋“ค์€ ๋ณดํ†ต Long Tailed..

[๋”ฅ๋Ÿฌ๋‹ ๋…ผ๋ฌธ๋ฆฌ๋ทฐ] Mixed Precision Training (ICLR 2018)

NVIDIA์™€ Baidu์—์„œ ์—ฐ๊ตฌํ•˜๊ณ  ICLR 2018์— ๋ฐœํ‘œ๋œ ๋…ผ๋ฌธ์ธ Mixed Precision Training์„ ๋ฐ”ํƒ•์œผ๋กœ ์ •๋ฆฌํ•œ ๊ธ€์ž…๋‹ˆ๋‹ค. ๋”ฅ๋Ÿฌ๋‹ ํ•™์Šต ๊ณผ์ •์—์„œ Mixed Precision์„ ์ด์šฉํ•˜์—ฌ GPU resource๋ฅผ ํšจ์œจ์ ์œผ๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค. (NVIDIA ๋ธ”๋กœ๊ทธ ์ •๋ฆฌ๊ธ€: developer.nvidia.com/blog/mixed-precision-training-deep-neural-networks/) Floating Point Format ์‹ค์ˆ˜๋ฅผ ์ปดํ“จํ„ฐ๋กœ ๋‚˜ํƒ€๋‚ด๋Š” ๋ฐฉ๋ฒ•์— ๊ณ ์ •์†Œ์ˆ˜์ (Fixed Pint) ๋ฐฉ์‹๊ณผ ๋ถ€๋™์†Œ์ˆ˜์ (Floating Point) ๋ฐฉ์‹์ด ์กด์žฌํ•ฉ๋‹ˆ๋‹ค. (๋ถ€๋™์†Œ์ˆ˜์  ๋ฐฉ์‹์€ ๋– ๋Œ์ด ์†Œ์ˆ˜์  ๋ฐฉ์‹์ด๋ผ๊ณ ๋„ ํ•œ๋‹ค๊ณ  ํ•ฉ๋‹ˆ๋‹ค. ๊ท€์—ฝ๋„ค์š”) ๊ณ ์ •์†Œ์ˆ˜์  ๋ฐฉ์‹์€ ์ •์ˆ˜๋ถ€์™€ ์†Œ์ˆ˜๋ถ€๋ฅผ ..

[Adversarial Example ๋…ผ๋ฌธ๋ฆฌ๋ทฐ] MagNet and "Efficient Defenses Against Adversarial Attacks

Carlini, Nicholas, and David Wagner. "Magnet and" efficient defenses against adversarial attacks" are not robust to adversarial examples." arXiv preprint arXiv:1711.08478 (2017). - 2017๋…„ Carlini์™€ Wagner์ด ์ œ์•ˆํ•œ MagNet๊ณผ "Efficient Defenses~"์˜ ๋ฐ˜๋ฐ• ๋…ผ๋ฌธ.- 1) Meng et al.์˜ MagNet (2017), 2) Zantedeschi et al.์˜ "Efficient Defenses Against Adversarial Attacks" (2017), 3) Shen et al.์˜ APE-GAN (2017) 3๊ฐ€์ง€์˜ adve..

[Adversarial Example ๋…ผ๋ฌธ๋ฆฌ๋ทฐ] DAPAS : Denoising Autoencoder to Prevent Adversarial attac

Cho, Seung Ju, et al. "DAPAS: Denoising Autoencoder to Prevent Adversarial attack in Semantic Segmentation." arXiv preprint arXiv:1908.05195 (2019). Abstract - Denoise autoencoder์„ ์ด์šฉํ•˜์—ฌ semantic segmentation์— ๋Œ€ํ•œ adversarial defense ๊ธฐ๋ฒ•์„ ์ œ์•ˆ Introduction - image๋ฅผ pixel level์—์„œ ์žฌ๊ตฌ์„ฑํ•˜์—ฌ ๊นจ๋—ํ•œ image๋ฅผ ๋งŒ๋“ ๋‹ค. - 1) Gaussian distribution, 2) Uniform distribution, 3) Bimodal distribution์„ noise๋กœ ์‚ฌ์šฉ. - Dataset : ..

[Adversarial Example ๋…ผ๋ฌธ๋ฆฌ๋ทฐ] MagNet

Meng, Dongyu, and Hao Chen. "Magnet: a two-pronged defense against adversarial examples." Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017. ์ด๋ฏธ์ง€ ์ถœ์ฒ˜ : https://www.youtube.com/watch?v=wZ-wIdAcWQE 2017๋…„์— ์ œ์•ˆ๋œ, autoencoder์„ ์ด์šฉํ•œ adversarial defense ๋ฐฉ๋ฒ•. ๊ฐœ์š” - Adversarial attack์— ๋Œ€ํ•œ defense ๋ฐฉ๋ฒ•. - Autoencoder์„ ์ด์šฉํ•˜์—ฌ "sanitize" ํ•œ input์„ classifier์— ๋„ฃ๋Š”๋‹ค. ์žฅ์  - target ..

๋ฐ˜์‘ํ˜•