FC2ブログ

[AI]Caffe

長ったらしく経緯を書いてるけど、内容まとめると以下の感じ。

1.Caffeインストール(WSL、1回目、失敗)
  WSLで進めたけど、import caffeでエラーになる。
2.最初からCaffeインストールやり直し(WSL、import caffeまで成功!)
  これをやってimport caffeまでできたけど、その後のMNISTによる学習準備でエラー。WSLの問題と予想し、VMwareへ。
3.WSLをやめてVMwareでやり直し(途中)
  今ここ。


1. Caffeインストール(WSL、1回目、失敗)

https://qiita.com/t_takii/items/de9d83b47baf6697ae6b

WSL(Windows Subsystem for Linux)のUbuntu18.04.2 LTS(カーネル4.4.0-17763-Microsoft)にて作業。

$ sudo apt-get update
$ sudo apt-get install -y --no-install-recommends libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev libatlas-base-dev libopenblas-dev protobuf-compiler
$ sudo apt-get install the python-dev

$ sudo apt install git
$ git clone https://github.com/BVLC/caffe.git

$ sudo apt-get install -y python-pip python-numpy python-scipy python-skimage python-protobuf

$ cd caffe
$ cp Makefile.config.example Makefile.config
$ vi Makefile.config

# 以下、編集箇所のみ赤字
CPU_ONLY := 1 ※コメントアウトを解除
PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/local/lib/python2.7/dist-packages/numpy/core/include
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/



$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.100.0.1 /usr/lib/x86_64-linux-gnu/libhdf5.so
$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.100.0.0 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so

$ sudo vi ~/.bashrc

# 最終行に追加
export PYTHONPATH=/home/bob/caffe/caffe/python/:$PYTHONPATH


$ source ~/.bashrc

CPU数確認
$ nproc
8

CPU数を指定してコンパイル
$ make all -j8

 :
&:build_release .) /``build_release'undefinedlibcvcv/
/::lib.referencelibcaffe.so::/build_release :imdecodeimdecodelibcaffe.so/to ((:lib undefinedcvcv /` ::undefinedlibcaffe.socvreference:: :: _InputArray_InputArrayreference :to undefinedimdecode constconstto (`&& referencecvcv,,` :: cvto::intint: _InputArrayimdecode)):` (''imdecodecvconstcv

(:&:collect2: error: ld returned 1 exit status
cv:,:collect2: error: ld returned 1 exit status
:imdecode _InputArray:(int _InputArraycv)const :'&const:
,&_InputArray , int const)int&')collect2: error: ld returned 1 exit status
,
'Makefile:635: recipe for target '.build_release/tools/compute_image_mean.bin' failed

make: *** [.build_release/tools/compute_image_mean.bin] Error 1
intcollect2: error: ld returned 1 exit status
Makefile:635: recipe for target '.build_release/tools/upgrade_solver_proto_text.bin' failed
)collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/upgrade_solver_proto_text.bin] Error 1
'Makefile:635: recipe for target '.build_release/tools/upgrade_net_proto_binary.bin' failed

make: *** [.build_release/tools/upgrade_net_proto_binary.bin] Error 1
collect2: error: ld returned 1 exit status
Makefile:635: recipe for target '.build_release/tools/convert_imageset.bin' failed
make: *** [.build_release/tools/convert_imageset.bin] Error 1
Makefile:635: recipe for target '.build_release/tools/caffe.bin' failed
make: *** [.build_release/tools/caffe.bin] Error 1
Makefile:635: recipe for target '.build_release/tools/extract_features.bin' failed
make: *** [.build_release/tools/extract_features.bin] Error 1



エラーを調べたら以下掲示板を発見。
https://github.com/BVLC/caffe/issues/2348
ここでこんな事言ってる。

aseempatni commented on 7 Sep 2015
Thanks @hongzhenwang. That helped.

Added LIBRARIES += glog gflags protobuf leveldb snappy \ lmdb boost_system hdf5_hl hdf5 m \ opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs to Makefile.config

I used make clean instead of rm -rf ./build

Then did make all and everything worked fine.



真似してやってみる。
$ vi Makefile.config
最終行に以下を追加。

LIBRARIES += glog gflags protobuf leveldb snappy lmdb boost_system hdf5_hl hdf5 m opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs


$ sudo make clean
$ sudo make all -j8
これで確かにエラーなく終わった。

 :
CXX examples/mnist/convert_mnist_data.cpp
CXX examples/siamese/convert_mnist_siamese_data.cpp
AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so.1.0.0
CXX/LD -o .build_release/tools/caffe.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX/LD -o .build_release/tools/upgrade_solver_proto_text.bin
CXX/LD -o .build_release/examples/cifar10/convert_cifar_data.bin
CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin
CXX/LD -o .build_release/examples/siamese/convert_mnist_siamese_data.bin
CXX/LD -o .build_release/examples/cpp_classification/classification.bin



$ sudo make test -j8

 :
LD .build_release/src/caffe/test/test_softmax_with_loss_layer.o
LD .build_release/src/caffe/test/test_solver.o
LD .build_release/src/caffe/test/test_solver_factory.o
LD .build_release/src/caffe/test/test_split_layer.o
LD .build_release/src/caffe/test/test_spp_layer.o
LD .build_release/src/caffe/test/test_stochastic_pooling.o
LD .build_release/src/caffe/test/test_syncedmem.o
LD .build_release/src/caffe/test/test_tanh_layer.o
LD .build_release/src/caffe/test/test_tile_layer.o
LD .build_release/src/caffe/test/test_threshold_layer.o
LD .build_release/src/caffe/test/test_upgrade_proto.o
LD .build_release/src/caffe/test/test_util_blas.o



$ sudo make runtest -j8
こんな感じで正常に終わったみたい。

 :
[ OK ] CropLayerTest/0.TestCropAll (0 ms)
[ RUN ] CropLayerTest/0.TestCropAllGradient
[ OK ] CropLayerTest/0.TestCropAllGradient (110 ms)
[----------] 11 tests from CropLayerTest/0 (969 ms total)

[----------] 3 tests from TanHLayerTest/0, where TypeParam = caffe::CPUDevice
[ RUN ] TanHLayerTest/0.TestTanHGradient
[ OK ] TanHLayerTest/0.TestTanHGradient (2 ms)
[ RUN ] TanHLayerTest/0.TestTanH
[ OK ] TanHLayerTest/0.TestTanH (0 ms)
[ RUN ] TanHLayerTest/0.TestTanHOverflow
[ OK ] TanHLayerTest/0.TestTanHOverflow (0 ms)
[----------] 3 tests from TanHLayerTest/0 (6 ms total)

[----------] Global test environment tear-down
[==========] 1162 tests from 152 test cases ran. (36402 ms total)
[ PASSED ] 1162 tests.




PyCaffe化(WSL、1回目、失敗)

ここからはこちらを参考。
https://qiita.com/t-hiroyoshi/items/3bba01dd11b1241f1336

runtestが通ったのでCaffeのbuildは成功。
次にこれをPyCaffeできるようにする。
以下でPyCaffeに必要なライブラリをpipで一気に入れる。
$ cd python
$ for li in $(cat requirements.txt); do sudo pip install $li; done
こんな感じで終わった。

 :
Collecting python-gflags>=2.0
Downloading https://files.pythonhosted.org/packages/df/ec/e31302d355bcb9d207d9b858adc1ecc4a6d8c855730c8ba4ddbdd3f8eb8d/python-gflags-3.1.2.tar.gz (52kB)
100% |████████████████████████████████| 61kB 529kB/s
Installing collected packages: python-gflags
Running setup.py install for python-gflags ... done
Successfully installed python-gflags-3.1.2
The directory '/home/bob/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/bob/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: pyyaml>=3.10 in /usr/lib/python2.7/dist-packages
The directory '/home/bob/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/bob/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: Pillow>=2.3.0 in /usr/lib/python2.7/dist-packages
The directory '/home/bob/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/bob/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: six>=1.1.0 in /home/bob/.local/lib/python2.7/site-packages



次はPyCaffeをbuild
xxx@x280:~/caffe/caffe/python$ cd ../
xxx@x280:~/caffe/caffe$ make pycaffe

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
touch python/caffe/proto/__init__.py
touch: cannot touch 'python/caffe/proto/__init__.py': Permission denied
Makefile:657: recipe for target 'python/caffe/proto/__init__.py' failed
make: *** [python/caffe/proto/__init__.py] Error 1



$ sudo make pycaffe
エラー出たのは単にsudoが足りなかったらしい。

touch python/caffe/proto/__init__.py
PROTOC (python) src/caffe/proto/caffe.proto



$ sudo make distribute

# add proto
cp -r src/caffe/proto distribute/
# add include
cp -r include distribute/
mkdir -p distribute/include/caffe/proto
cp .build_release/src/caffe/proto/caffe.pb.h distribute/include/caffe/proto
# add tool and example binaries
cp .build_release/tools/caffe.bin .build_release/tools/compute_image_mean.bin .build_release/tools/convert_imageset.bin .build_release/tools/extract_features.bin .build_release/tools/upgrade_net_proto_binary.bin .build_release/tools/upgrade_net_proto_text.bin .build_release/tools/upgrade_solver_proto_text.bin distribute/bin
cp .build_release/examples/cifar10/convert_cifar_data.bin .build_release/examples/cpp_classification/classification.bin .build_release/examples/mnist/convert_mnist_data.bin .build_release/examples/siamese/convert_mnist_siamese_data.bin distribute/bin
# add libraries
cp .build_release/lib/libcaffe.a distribute/lib
install -m 644 .build_release/lib/libcaffe.so.1.0.0 distribute/lib
cd distribute/lib; rm -f libcaffe.so; ln -s libcaffe.so.1.0.0 libcaffe.so
# add python - it's not the standard way, indeed...
cp -r python distribute/




$ python
Python 2.7.15+ (default, Nov 27 2018, 23:36:35)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Pythonインタプリタを起動した後、import caffeでエラーなく終了すれば動作成功との事だが、エラー出た。

Traceback (most recent call last):
File "", line 1, in
File "/home/bob/caffe/caffe/python/caffe/__init__.py", line 1, in
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File "/home/bob/caffe/caffe/python/caffe/pycaffe.py", line 15, in
import caffe.io
File "/home/bob/caffe/caffe/python/caffe/io.py", line 2, in
import skimage.io
File "/usr/lib/python2.7/dist-packages/skimage/__init__.py", line 158, in
from .util.dtype import *
File "/usr/lib/python2.7/dist-packages/skimage/util/__init__.py", line 7, in
from .arraycrop import crop
File "/usr/lib/python2.7/dist-packages/skimage/util/arraycrop.py", line 8, in
from numpy.lib.arraypad import _validate_lengths
ImportError: cannot import name _validate_lengths





2. 最初からCaffeインストールやり直し(WSL、import caffeまで成功!)

https://stackoverflow.com/questions/54603645/caffe-import-error-even-after-installing-it-successfully-on-ubuntu-18-04
上のエラーに出会ったやつは、ここの記事で以下の手順で試せとある。


1 Answer
Problem solved:  xxxx
xxx
Let me give step by step guidance to install caffe successfully in Ubuntu 18.04 LTS:
1] sudo apt-get install -y --no-install-recommends libboost-all-dev
 :



試してみる。

$ sudo apt-get install -y --no-install-recommends libboost-all-dev
$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler
# git clone https://github.com/BVLC/caffe
# cd caffe
# cp Makefile.config.example Makefile.config
pipが無い場合はインストールしておく。
# sudo apt install python-pip python3-pip
# sudo pip install scikit-image protobuf
# cd python
# for req in $(cat requirements.txt); do sudo pip install $req; done
# cd ../
# vi Makefile.config
CPU_ONLY := 1 --> uncomment
OPENCV_VERSION := 3. --> uncomment
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5 opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs --> add to last line

$ sudo make all
このエラーは想定内らしい。

 :
CXX src/caffe/layers/flatten_layer.cpp
CXX src/caffe/layers/hdf5_data_layer.cpp
src/caffe/layers/hdf5_data_layer.cpp:14:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^~~~~~~~
compilation terminated.
Makefile:591: recipe for target '.build_release/src/caffe/layers/hdf5_data_layer.o' failed
make: *** [.build_release/src/caffe/layers/hdf5_data_layer.o] Error 1



$ sudo apt-get install libhdf5-dev
$ vi Makefile.config

赤字を追加
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial



$ sudo make all
確かにエラーが消えた。

 :
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX tools/upgrade_solver_proto_text.cpp
CXX/LD -o .build_release/tools/upgrade_solver_proto_text.bin
CXX examples/cifar10/convert_cifar_data.cpp
CXX/LD -o .build_release/examples/cifar10/convert_cifar_data.bin
CXX examples/cpp_classification/classification.cpp
CXX/LD -o .build_release/examples/cpp_classification/classification.bin
CXX examples/mnist/convert_mnist_data.cpp
CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin
CXX examples/siamese/convert_mnist_siamese_data.cpp
CXX/LD -o .build_release/examples/siamese/convert_mnist_siamese_data.bin



$ sudo make test
$ sudo make runtest
$ sudo make pycaffe

掲示板の手順ではmake pycaffeでエラー出るかも、とある。
特にエラーは出なかったが、1回目のインストールの時と違う設定なので以下の通り修正しておく。
$ vi Makefile.config

PYTHON_INCLUDE := /usr/include/python2.7 /usr/local/lib/python2.7/dist-packages/numpy/core/include


$ source ~/.bashrc

これで再度import caffeを試す。
$ python
>> import caffe
さっきと同じく、こんなやつが出る。

 :
from numpy.lib.arraypad import _validate_lengths
ImportError: cannot import name '_validate_lengths'



調べると、skimage package.をアップグレードしたら解決したという記事あり。
https://stackoverflow.com/questions/54241226/importerror-cannot-import-name-validate-lengths

$ pip install --upgrade scikit-image
再度試してみたら、10秒ほどかかったが、正常にimport caffeができた(エラーなく終了した)。これで環境整備完了。
$ python
>>> import caffe
>>>


MNIST学習(WSLだと失敗)

https://qiita.com/t-hiroyoshi/items/2bf473fd06c352d97579

WSL起動。
$ cd /home/bob/caffe/caffe
$ sudo ./data/mnist/get_mnist.sh

Downloading...
--2019-07-05 10:56:49-- http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
Resolving yann.lecun.com (yann.lecun.com)... 216.165.22.6
Connecting to yann.lecun.com (yann.lecun.com)|216.165.22.6|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9912422 (9.5M) [application/x-gzip]
Saving to: ‘train-images-idx3-ubyte.gz’

train-images-idx3-ubyte 100%[==============================>] 9.45M 1.60MB/s in 7.0s

2019-07-05 10:56:57 (1.35 MB/s) - ‘train-images-idx3-ubyte.gz’ saved [9912422/9912422]

--2019-07-05 10:56:57-- http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
Resolving yann.lecun.com (yann.lecun.com)... 216.165.22.6
Connecting to yann.lecun.com (yann.lecun.com)|216.165.22.6|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 28881 (28K) [application/x-gzip]
Saving to: ‘train-labels-idx1-ubyte.gz’

train-labels-idx1-ubyte 100%[==============================>] 28.20K 178KB/s in 0.2s

2019-07-05 10:56:58 (178 KB/s) - ‘train-labels-idx1-ubyte.gz’ saved [28881/28881]

--2019-07-05 10:56:58-- http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
Resolving yann.lecun.com (yann.lecun.com)... 216.165.22.6
Connecting to yann.lecun.com (yann.lecun.com)|216.165.22.6|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1648877 (1.6M) [application/x-gzip]
Saving to: ‘t10k-images-idx3-ubyte.gz’

t10k-images-idx3-ubyte. 100%[==============================>] 1.57M 793KB/s in 2.0s

2019-07-05 10:57:00 (793 KB/s) - ‘t10k-images-idx3-ubyte.gz’ saved [1648877/1648877]

--2019-07-05 10:57:00-- http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
Resolving yann.lecun.com (yann.lecun.com)... 216.165.22.6
Connecting to yann.lecun.com (yann.lecun.com)|216.165.22.6|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4542 (4.4K) [application/x-gzip]
Saving to: ‘t10k-labels-idx1-ubyte.gz’

t10k-labels-idx1-ubyte. 100%[==============================>] 4.44K --.-KB/s in 0s

2019-07-05 10:57:00 (110 MB/s) - ‘t10k-labels-idx1-ubyte.gz’ saved [4542/4542]



Caffeで画像認識させるために、先のダウンロードデータをLMDB形式に変換。
$ sudo ./examples/mnist/create_mnist.sh

Creating lmdb...
I0705 11:02:20.452775 55 db_lmdb.cpp:35] Opened lmdb examples/mnist/mnist_train_lmdb
I0705 11:02:20.454550 55 convert_mnist_data.cpp:88] A total of 60000 items.
I0705 11:02:20.454649 55 convert_mnist_data.cpp:89] Rows: 28 Cols: 28
F0705 11:02:20.480227 55 db_lmdb.hpp:15] Check failed: mdb_status == 0 (-30796 vs. 0) MDB_CORRUPTED: Located page was wrong type
*** Check failure stack trace: ***
@ 0x7f608b81c0cd google::LogMessage::Fail()
@ 0x7f608b81df33 google::LogMessage::SendToLog()
@ 0x7f608b81bc28 google::LogMessage::Flush()
@ 0x7f608b81e999 google::LogMessageFatal::~LogMessageFatal()
@ 0x7f608bd4323d caffe::db::LMDBTransaction::Commit()
@ 0x7f608c403189 convert_dataset()
@ 0x7f608c40247a main
@ 0x7f608a7f1b97 __libc_start_main
@ 0x7f608c4024ca _start
Aborted (core dumped)



今更だけど、CAFFE_ROOTを~/.bashrcへ追加しても結果変わらず。
export CAFFE_ROOT=/home/bob/caffe/caffe

https://github.com/BVLC/caffe/issues/6578
ここ見ると、WSL使ってると同じエラーが出て、WSLでないUbuntuだと成功したって言ってる。




3. WSLをやめてVMwareでやり直し(import caffe成功、MNIST学習成功)

Ubuntu18.04.2 LTS (Bionic Beaver)
カーネル4.15.0-45-generic

$ sudo apt-get install -y --no-install-recommends libboost-all-dev
E: Unable to locate package なんとかでエラーで止まった。
$ sudo apt-get update
$ sudo apt-get install -y --no-install-recommends libboost-all-dev
パッケージ名が間違っている訳ではないので原因が不明だったが、アップデートしてから再度やってみたらエラー消えた。
$ sudo apt-get -y install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler
# git clone https://github.com/BVLC/caffe
# cd caffe
# cp Makefile.config.example Makefile.config
# sudo apt install python-pip python3-pip
# sudo pip install scikit-image protobuf
# cd python
# for req in $(cat requirements.txt); do sudo pip install $req; done
# cd ../
# vi Makefile.config
CPU_ONLY := 1 --> uncomment
OPENCV_VERSION := 3. --> uncomment
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5 opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs --> add to last line

$ sudo make all > 失敗する

 :
src/caffe/layers/hdl5_data_layer.cpp:14:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
~~~~~~~~~
compilation terminated.
Makefile:591: recipe for target '.build_release/src/caffe/layers/hdf5_data_layer.o' failed
make: *** [.build_release/src/caffe/layers/hdf5_data_layer.o] Error 1



$ sudo apt-get install libhdf5-dev
$ vi Makefile.config

赤字を追加
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial



$ sudo make all
$ sudo make test
$ sudo make runtest
$ sudo make pycaffe

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
python/caffe/_caffe.cpp:10:10: fatal error: numpy/arrayobject.h: No such file or directory
#include
~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:517: recipe for target 'python/caffe/_caffe.so' failed
make: *** [python/caffe/_caffe.so] Error 1



arrayobject.hが無いというので探してみたら、
/usr/local/lib/python2.7/dist-packages/numpy/core/include/numpy/arrayobject.h
にあった。Makefile.configのパスが間違ってた。
$ sudo vi /home/bob/caffe/Makefile.config

PYTHON_INCLUDE := /usr/include/python2.7 \
修正前 /usr/lib/python2.7/dist-packages/numpy/core/include
修正後 /usr/local/lib/python2.7/dist-packages/numpy/core/include



$ sudo make pycaffe

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
touch python/caffe/proto/__init__.py
PROTOC (python) src/caffe/proto/caffe.proto


今度は成功した。

$ python
>>> import caffe

Traceback (most recent call last):
File "", line 1, in
ImportError: No module named caffe



こんなエラーが出た。原因はbashrcにパスが無いかららしい。
なので以下を追記。
$ vi ~/.bashrc

export PYTHONPATH=/home/bob/caffe/python



$ python
>>> import caffe
>>>
今度は成功した。

$
$ ./examples/mnist/create_mnist.sh

Creating lmdb...
I0706 11:14:06.001570 78478 db_lmdb.cpp:35] Opened lmdb examples/mnist/mnist_train_lmdb
I0706 11:14:06.005759 78478 convert_mnist_data.cpp:88] A total of 60000 items.
I0706 11:14:06.006155 78478 convert_mnist_data.cpp:89] Rows: 28 Cols: 28
I0706 11:14:06.978487 78478 convert_mnist_data.cpp:108] Processed 60000 items.
I0706 11:14:07.033179 78479 db_lmdb.cpp:35] Opened lmdb examples/mnist/mnist_train_lmdb
I0706 11:14:07.034142 78479 convert_mnist_data.cpp:88] A total of 10000 items.
I0706 11:14:07.034878 78479 convert_mnist_data.cpp:89] Rows: 28 Cols: 28
I0706 11:14:07.220436 78479 convert_mnist_data.cpp:108] Processed 10000 items.
Done.



$ sudo ./examples/mnist/train_lenet.sh

I0706 11:28:43.753690 78506 caffe.cpp:204] Using GPUs 0
F0706 11:28:43.754091 78506 common.cpp:66] Cannot use GPU in CPU-only Caffe; check more.
*** Check failure stack trace: ***
@ 0x7f19xxxxxx google::LogMessage::Fail()
 :
Aborted (core dumped)


CPU-onlyである事を忘れてた。

$ sudo vi ./examples/mnist/lenet_solver.prototxt

solver_mode: CPU ※ GPUからCPUへ変更



これで再度トレーニングを実施。
$ sudo ./examples/mnist/train_lenet.sh

 :
I0706 11:50:35.xxxxx 78590 solver.cpp:347] Iteration 10000, loss = 0.00282926
I0706 11:50:35.xxxxx 78590 solver.cpp:347] Iteration 10000, Testing net (#0)
I0706 11:50:35.xxxxx 78590 data_layer.cpp:73] Restarting data prefetching from start.
I0706 11:50:35.xxxxx 78590 solver.cpp:414] Test net output #0: accuracy = 0.99
I0706 11:50:35.xxxxx 78590 solver.cpp:414] Test net output #1: loss = 0.028678 (* 1 = 0.028678 loss)
I0706 11:50:35.xxxxx 78590 solver.cpp:250] Optimization Done.
I0706 11:50:35.xxxxx 78590 caffe.cpp:250] Optimization Done.


だいたい10分くらいで完了かな。

https://qiita.com/t-hiroyoshi/items/2bf473fd06c352d97579

これで、.examples/mnist/lenet_iter_10000.caffemodelが出力された学習済モデル。
これを読み込めば、MNISTの学習後の状態を再現できる。
スポンサーサイト



カレンダー
06 | 2019/07 | 08
- 1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31 - - -
累積訪問者
現在の訪問者
現在の閲覧者数:
最新記事
最新トラックバック
最新コメント
月別アーカイブ
カテゴリ
プロフィール

bobgosso

Author:bobgosso
FPGAのブログへようこそ!

検索フォーム
RSSリンクの表示
リンク
ブロとも申請フォーム

この人とブロともになる

QRコード
QRコード