TensorFlow

since 2015-11-10

リンク

https://tensorflow.googlesource.com/tensorflow/
https://github.com/tensorflow/tensorflow/issues
http://googledevjp.blogspot.jp/2015/11/tensorflow-google.html
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- http://download.tensorflow.org/paper/whitepaper2015.pdf
http://www.tensorflow.org/get_started/os_setup.md

最後まで読まないで自分がやってしまった手順をメモ：

$ virtualenv py
New python executable in py/bin/python
Installing setuptools, pip...done.

$ source py/bin/activate

$ which python
/Users/nishimotz/work/tensorFlow/py/bin/python

$ python -V
Python 2.7.10

$ which pip
/Users/nishimotz/work/tensorFlow/py/bin/pip

$ pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl

$ python
Python 2.7.10 (default, Jul 14 2015, 19:46:27) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
can't determine number of CPU cores: assuming 4
I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 4
can't determine number of CPU cores: assuming 4
I tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 4
>>> print sess.run(hello)
Hello, TensorFlow!
>>> a = tf.constant(10)
>>> b = tf.constant(32)
>>> print sess.run(a+b)
42

Windows

since 2017-04-16

ひさしぶりに触る。

Windows Python 3.5.3 (32bit) がたまたま入っている。

https://www.tensorflow.org/install/install_windows

venv の説明はこちら http://bootcamp-text.readthedocs.io/textbook/6_pyvenv.html

> py -3.5 -m venv env
> env\Scripts\activate
> python -V
Python 3.5.3
> pip install --upgrade tensorflow
Collecting tensorflow
  Could not find a version that satisfies the requirement tensorflow (from versions: )
No matching distribution found for tensorflow

ないじゃん。。x64 でないからかな。。

Windows Python 3.5.3 (AMD64) を追加インストール。

> py -3.5 -m venv env
> env\Scripts\activate
> python -V
Python 3.5.3
> pip install --upgrade tensorflow
Collecting tensorflow
  Downloading tensorflow-1.0.1-cp35-cp35m-win_amd64.whl (14.7MB)
    100% |################################| 14.7MB 62kB/s
Collecting protobuf>=3.1.0 (from tensorflow)
  Downloading protobuf-3.2.0-py2.py3-none-any.whl (360kB)
    100% |################################| 368kB 858kB/s
Collecting wheel>=0.26 (from tensorflow)
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |################################| 71kB 1.3MB/s
Collecting numpy>=1.11.0 (from tensorflow)
  Downloading numpy-1.12.1-cp35-none-win_amd64.whl (7.7MB)
    100% |################################| 7.7MB 144kB/s
Collecting six>=1.10.0 (from tensorflow)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting setuptools (from protobuf>=3.1.0->tensorflow)
  Downloading setuptools-35.0.0-py2.py3-none-any.whl (390kB)
    100% |################################| 399kB 936kB/s
Collecting packaging>=16.8 (from setuptools->protobuf>=3.1.0->tensorflow)
  Using cached packaging-16.8-py2.py3-none-any.whl
Collecting appdirs>=1.4.0 (from setuptools->protobuf>=3.1.0->tensorflow)
  Downloading appdirs-1.4.3-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools->protobuf>=3.1.0->tensorflow)
  Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
    100% |################################| 61kB 2.4MB/s
Installing collected packages: six, pyparsing, packaging, appdirs, setuptools, protobuf, wheel, numpy, tensorflow
  Found existing installation: setuptools 28.8.0
    Uninstalling setuptools-28.8.0:
      Successfully uninstalled setuptools-28.8.0
Successfully installed appdirs-1.4.3 numpy-1.12.1 packaging-16.8 protobuf-3.2.0 pyparsing-2.2.0 setuptools-35.0.0 six-1.10.0 tensorflow-1.0.1 wheel-0.29.0

うまくいった。

> pip install -U tensorflow-gpu
Collecting tensorflow-gpu
  Downloading tensorflow_gpu-1.0.1-cp35-cp35m-win_amd64.whl (43.1MB)
    100% |################################| 43.1MB 23kB/s
Requirement already up-to-date: numpy>=1.11.0 in c:\work\2017\tf\env\lib\site-packages (from tensorflow-gpu)
Requirement already up-to-date: six>=1.10.0 in c:\work\2017\tf\env\lib\site-packages (from tensorflow-gpu)
Requirement already up-to-date: wheel>=0.26 in c:\work\2017\tf\env\lib\site-packages (from tensorflow-gpu)
Requirement already up-to-date: protobuf>=3.1.0 in c:\work\2017\tf\env\lib\site-packages (from tensorflow-gpu)
Requirement already up-to-date: setuptools in c:\work\2017\tf\env\lib\site-packages (from protobuf>=3.1.0->tensorflow-gpu)
Requirement already up-to-date: appdirs>=1.4.0 in c:\work\2017\tf\env\lib\site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu)
Requirement already up-to-date: packaging>=16.8 in c:\work\2017\tf\env\lib\site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu)
Requirement already up-to-date: pyparsing in c:\work\2017\tf\env\lib\site-packages (from packaging>=16.8->setuptools->protobuf>=3.1.0->tensorflow-gpu)
Installing collected packages: tensorflow-gpu
Successfully installed tensorflow-gpu-1.0.1

こっちも入った。

しかし、対応ハードウェアが必要だったらしい。。

> python
Python 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:126] Couldn't open CUDA library cublas64_80.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_blas.cc:2294] Unable to load cuBLAS DSO.
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:126] Couldn't open CUDA library cudnn64_5.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_dnn.cc:3517] Unable to load cuDNN DSO
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:126] Couldn't open CUDA library cufft64_80.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_fft.cc:344] Unable to load cuFFT DSO.
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:126] Couldn't open CUDA library nvcuda.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_diagnostics.cc:165] hostname: ・誹・
Traceback (most recent call last):
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 18, in swig_import_helper
    return importlib.import_module(mname)
  File "C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 577, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 914, in create_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\__init__.py", line 66, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 21, in <module>
    _pywrap_tensorflow = swig_import_helper()
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 20, in swig_import_helper
    return importlib.import_module('_pywrap_tensorflow')
  File "C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: No module named '_pywrap_tensorflow'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\__init__.py", line 72, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 18, in swig_import_helper
    return importlib.import_module(mname)
  File "C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 577, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 914, in create_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\__init__.py", line 66, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 21, in <module>
    _pywrap_tensorflow = swig_import_helper()
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 20, in swig_import_helper
    return importlib.import_module('_pywrap_tensorflow')
  File "C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: No module named '_pywrap_tensorflow'


Failed to load the native TensorFlow runtime.

See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#import_error

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

けっきょく pip uninstall tensorflow-gpu tensorflow して、インストールからやり直す。

>  python
Python 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "BestSplits" device_type: "CPU"') for unknown op: BestSplits
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "CountExtremelyRandomStats" device_type: "CPU"') for unknown op: CountExtremelyRandomStats
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "FinishedNodes" device_type: "CPU"') for unknown op: FinishedNodes
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "GrowTree" device_type: "CPU"') for unknown op: GrowTree
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ReinterpretStringToFloat" device_type: "CPU"') for unknown op: ReinterpretStringToFloat
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "SampleInputs" device_type: "CPU"') for unknown op: SampleInputs
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ScatterAddNdim" device_type: "CPU"') for unknown op: ScatterAddNdim
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNInsert" device_type: "CPU"') for unknown op: TopNInsert
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNRemove" device_type: "CPU"') for unknown op: TopNRemove
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TreePredictions" device_type: "CPU"') for unknown op: TreePredictions
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "UpdateFertileSlots" device_type: "CPU"') for unknown op: UpdateFertileSlots
b'Hello, TensorFlow!'

まあいいんだろう、これで。。

tf-seq2seq

seq2seq もためす。

https://google.github.io/seq2seq/getting_started/

> git clone https://github.com/google/seq2seq.git
Cloning into 'seq2seq'...
remote: Counting objects: 5965, done.
remote: Compressing objects: 100% (48/48), done.
remote: Total 5965 (delta 22), reused 0 (delta 0), pack-reused 5917
Receiving objects: 100% (5965/5965), 1.62 MiB | 92.00 KiB/s, done.
Resolving deltas: 100% (4158/4158), done.

> cd seq2seq

> pip install -e .
Obtaining file:///C:/work/2017/tf/seq2seq
Requirement already satisfied: numpy in c:\work\2017\tf\env\lib\site-packages (from seq2seq==0.1)
Collecting matplotlib (from seq2seq==0.1)
  Downloading matplotlib-2.0.0-cp35-cp35m-win_amd64.whl (9.0MB)
    100% |################################| 9.0MB 96kB/s
Collecting pyyaml (from seq2seq==0.1)
  Downloading PyYAML-3.12-cp35-cp35m-win_amd64.whl (195kB)
    100% |################################| 204kB 823kB/s
Collecting pyrouge (from seq2seq==0.1)
  Downloading pyrouge-0.1.3.tar.gz (60kB)
    100% |################################| 61kB 712kB/s
Collecting python-dateutil (from matplotlib->seq2seq==0.1)
  Downloading python_dateutil-2.6.0-py2.py3-none-any.whl (194kB)
    100% |################################| 194kB 474kB/s
Collecting cycler>=0.10 (from matplotlib->seq2seq==0.1)
  Downloading cycler-0.10.0-py2.py3-none-any.whl
Collecting pytz (from matplotlib->seq2seq==0.1)
  Downloading pytz-2017.2-py2.py3-none-any.whl (484kB)
    100% |################################| 491kB 391kB/s
Requirement already satisfied: six>=1.10 in c:\work\2017\tf\env\lib\site-packages (from matplotlib->seq2seq==0.1)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 in c:\work\2017\tf\env\lib\site-packages (from matplotlib->seq2seq==0.1)
Building wheels for collected packages: pyrouge
  Running setup.py bdist_wheel for pyrouge ... done
  Stored in directory: C:\Users\nishimotz\AppData\Local\pip\Cache\wheels\3c\e1\54\9e59592032c08e38e9f700f3b430a06e02eb6fa77e699d02ce
Successfully built pyrouge
Installing collected packages: python-dateutil, cycler, pytz, matplotlib, pyyaml, pyrouge, seq2seq
  Running setup.py develop for seq2seq
Successfully installed cycler-0.10.0 matplotlib-2.0.0 pyrouge-0.1.3 python-dateutil-2.6.0 pytz-2017.2 pyyaml-3.12 seq2seq

テスト

> python -m unittest seq2seq.test.pipeline_test
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "BestSplits" device_type: "CPU"') for unknown op: BestSplits
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "CountExtremelyRandomStats" device_type: "CPU"') for unknown op: CountExtremelyRandomStats
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "FinishedNodes" device_type: "CPU"') for unknown op: FinishedNodes
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "GrowTree" device_type: "CPU"') for unknown op: GrowTree
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ReinterpretStringToFloat" device_type: "CPU"') for unknown op: ReinterpretStringToFloat
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "SampleInputs" device_type: "CPU"') for unknown op: SampleInputs
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ScatterAddNdim" device_type: "CPU"') for unknown op: ScatterAddNdim
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNInsert" device_type: "CPU"') for unknown op: TopNInsert
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNRemove" device_type: "CPU"') for unknown op: TopNRemove
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TreePredictions" device_type: "CPU"') for unknown op: TreePredictions
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "UpdateFertileSlots" device_type: "CPU"') for unknown op: UpdateFertileSlots
.INFO:tensorflow:Loading config from C:\Users\NISHIM~1\AppData\Local\Temp\tmpa_i3l_wj\train_config.yml
INFO:tensorflow:Final Config:
input_pipeline_dev:
  class: ParallelTextInputPipeline
  params:
    source_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmp2ld4u5ss']
    target_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmpuo0hqt9l']
input_pipeline_train:
  class: ParallelTextInputPipeline
  params:
    source_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmp79bixs0q']
    target_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmphefrqhe0']
model_params:
  decoder.params:
    rnn_cell:
      cell_class: GRUCell
      cell_params: {num_units: 8}
  embedding.dim: 10
  encoder.params:
    rnn_cell:
      cell_class: GRUCell
      cell_params: {num_units: 8}
train_steps: 50

INFO:tensorflow:Setting save_checkpoints_secs to 600
INFO:tensorflow:Creating ParallelTextInputPipeline in mode=train
INFO:tensorflow:
ParallelTextInputPipeline:
  num_epochs: null
  shuffle: true
  source_delimiter: ' '
  source_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmp79bixs0q']
  target_delimiter: ' '
  target_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmphefrqhe0']

INFO:tensorflow:Creating ParallelTextInputPipeline in mode=eval
INFO:tensorflow:
ParallelTextInputPipeline:
  num_epochs: 1
  shuffle: false
  source_delimiter: ' '
  source_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmp2ld4u5ss']
  target_delimiter: ' '
  target_files: ['C:\Users\NISHIM~1\AppData\Local\Temp\tmpuo0hqt9l']

INFO:tensorflow:Using config: {'_keep_checkpoint_every_n_hours': 4, '_task_type': None, '_evaluation_master': '', '_master': '', '_environment': 'local', '_save_checkpoints_secs': 600, '_save_summary_steps': 100, '_task_id': 0, '_tf_random_seed': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x00000214D1DFEB38>, '_save_checkpoints_steps': None, '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1.0
}
, '_keep_checkpoint_max': 5, '_num_ps_replicas': 0, '_is_chief': True}
C:\work\2017\tf\env\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py:181: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead
  return inspect.getargspec(func).args
INFO:tensorflow:Creating PrintModelAnalysisHook in mode=train
INFO:tensorflow:
PrintModelAnalysisHook: {}

INFO:tensorflow:Creating MetadataCaptureHook in mode=train
INFO:tensorflow:
MetadataCaptureHook: {step: 10}

INFO:tensorflow:Creating TrainSampleHook in mode=train
INFO:tensorflow:
TrainSampleHook: {every_n_secs: null, every_n_steps: 1000, source_delimiter: ' ',
  target_delimiter: ' '}

INFO:tensorflow:Creating LogPerplexityMetricSpec in mode=eval
INFO:tensorflow:
LogPerplexityMetricSpec: {}

INFO:tensorflow:Creating BleuMetricSpec in mode=eval
INFO:tensorflow:
BleuMetricSpec: {eos_token: SEQUENCE_END, postproc_fn: '', separator: ' ', sos_token: SEQUENCE_START}

INFO:tensorflow:Creating RougeMetricSpec in mode=eval
INFO:tensorflow:
RougeMetricSpec: {eos_token: SEQUENCE_END, postproc_fn: '', rouge_type: rouge_1/f_score,
  separator: ' ', sos_token: SEQUENCE_START}

WARNING:tensorflow:From C:\work\2017\tf\env\lib\site-packages\tensorflow\contrib\learn\python\learn\monitors.py:267: BaseMonitor.__init__ (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.
INFO:tensorflow:Creating AttentionSeq2Seq in mode=train
INFO:tensorflow:
AttentionSeq2Seq:
  attention.class: AttentionLayerBahdanau
  attention.params: {num_units: 10}
  bridge.class: seq2seq.models.bridges.ZeroBridge
  bridge.params: {}
  decoder.class: seq2seq.decoders.AttentionDecoder
  decoder.params:
    rnn_cell:
      cell_class: GRUCell
      cell_params: {num_units: 8}
  embedding.dim: 10
  embedding.init_scale: 0.04
  embedding.share: false
  encoder.class: seq2seq.encoders.BidirectionalRNNEncoder
  encoder.params:
    rnn_cell:
      cell_class: GRUCell
      cell_params: {num_units: 8}
  inference.beam_search.beam_width: 0
  inference.beam_search.choose_successors_fn: choose_top_k
  inference.beam_search.length_penalty_weight: 0.0
  optimizer.clip_embed_gradients: 0.1
  optimizer.clip_gradients: 5.0
  optimizer.learning_rate: 0.0001
  optimizer.lr_decay_rate: 0.99
  optimizer.lr_decay_steps: 100
  optimizer.lr_decay_type: ''
  optimizer.lr_min_learning_rate: 1.0e-12
  optimizer.lr_staircase: false
  optimizer.lr_start_decay_at: 0
  optimizer.lr_stop_decay_at: 2147483647
  optimizer.name: Adam
  optimizer.params: {}
  optimizer.sync_replicas: 0
  optimizer.sync_replicas_to_aggregate: 0
  source.max_seq_len: 50
  source.reverse: true
  target.max_seq_len: 50
  vocab_source: C:\Users\NISHIM~1\AppData\Local\Temp\tmp2s79yt9f
  vocab_target: C:\Users\NISHIM~1\AppData\Local\Temp\tmpz6_mvzmb

INFO:tensorflow:Creating vocabulary lookup table of size 7
INFO:tensorflow:Creating vocabulary lookup table of size 7
INFO:tensorflow:Creating BidirectionalRNNEncoder in mode=train
INFO:tensorflow:
BidirectionalRNNEncoder:
  init_scale: 0.04
  rnn_cell:
    cell_class: GRUCell
    cell_params: {num_units: 8}
    dropout_input_keep_prob: 1.0
    dropout_output_keep_prob: 1.0
    num_layers: 1
    residual_combiner: add
    residual_connections: false
    residual_dense: false

C:\work\2017\tf\seq2seq\seq2seq\training\utils.py:118: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead
  cell_args = set(inspect.getargspec(cell_class.__init__).args[1:])
INFO:tensorflow:Creating AttentionLayerBahdanau in mode=train
INFO:tensorflow:
AttentionLayerBahdanau: {num_units: 10}

INFO:tensorflow:Creating AttentionDecoder in mode=train
INFO:tensorflow:
AttentionDecoder:
  init_scale: 0.04
  max_decode_length: 100
  rnn_cell:
    cell_class: GRUCell
    cell_params: {num_units: 8}
    dropout_input_keep_prob: 1.0
    dropout_output_keep_prob: 1.0
    num_layers: 1
    residual_combiner: add
    residual_connections: false
    residual_dense: false

INFO:tensorflow:Creating ZeroBridge in mode=train
INFO:tensorflow:
ZeroBridge: {}

INFO:tensorflow:Create CheckpointSaverHook.
EC:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\unittest\case.py:629: ResourceWarning: unclosed file <_io.BufferedRandom name=3>
  outcome.errors.clear()
C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\unittest\case.py:629: ResourceWarning: unclosed file <_io.BufferedRandom name=4>
  outcome.errors.clear()
C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\unittest\case.py:629: ResourceWarning: unclosed file <_io.BufferedRandom name=5>
  outcome.errors.clear()
C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\unittest\case.py:629: ResourceWarning: unclosed file <_io.BufferedRandom name=6>
  outcome.errors.clear()
C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\unittest\case.py:629: ResourceWarning: unclosed file <_io.BufferedRandom name=7>
  outcome.errors.clear()
C:\Users\nishimotz\AppData\Local\Programs\Python\Python35\lib\unittest\case.py:629: ResourceWarning: unclosed file <_io.BufferedRandom name=8>
  outcome.errors.clear()

======================================================================
ERROR: test_train_infer (seq2seq.test.pipeline_test.PipelineTest)
Tests training and inference scripts.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\work\2017\tf\seq2seq\seq2seq\test\pipeline_test.py", line 148, in test_train_infer
    train_script.main([])
  File "C:\work\2017\tf\seq2seq\bin\train.py", line 271, in main
    schedule=FLAGS.schedule)
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\contrib\learn\python\learn\learn_runner.py", line 106, in run
    return task()
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 459, in train_and_evaluate
    self.train(delay_secs=0)
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\contrib\learn\python\learn\experiment.py", line 281, in train
    monitors=self._train_monitors + extra_hooks)
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\util\deprecation.py", line 280, in new_func
    return func(*args, **kwargs)
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 426, in fit
    loss = self._train_model(input_fn=input_fn, hooks=hooks)
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 981, in _train_model
    config=self.config.tf_config) as mon_sess:
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 315, in MonitoredTrainingSession
    return MonitoredSession(session_creator=session_creator, hooks=all_hooks)
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 601, in __init__
    session_creator, hooks, should_recover=True)
  File "C:\work\2017\tf\env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 428, in __init__
    h.begin()
  File "C:\work\2017\tf\seq2seq\seq2seq\training\hooks.py", line 246, in begin
    opts = tf.contrib.tfprof.model_analyzer.TRAINABLE_VARS_PARAMS_STAT_OPTIONS
AttributeError: module 'tensorflow.contrib.tfprof' has no attribute 'model_analyzer'

----------------------------------------------------------------------
Ran 2 tests in 7.095s

FAILED (errors=1)

エラーになった。さて。。

https://github.com/google/seq2seq/issues/159

ここはまだ Windows に対応してないという話。

Bash on Ubuntu on Windows で試す

python3.5 は自分で make install していた。

$ which python3.5
/usr/local/bin/python3.5
$ python3.5 -m venv env-linux
$ . env-linux/bin/activate
(env-linux) nishimotz@LAPTOP-SH8BQD3I:/mnt/c/work/2017/tf$ pip install -U tensorflow
Collecting tensorflow
  Downloading tensorflow-1.0.1-cp35-cp35m-manylinux1_x86_64.whl (44.1MB)

中略。 seq2seq の unittest の結果

----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/c/work/2017/tf/seq2seq/seq2seq/test/pipeline_test.py", line 185, in test_train_infer
    infer_script.main([])
  File "/mnt/c/work/2017/tf/seq2seq/bin/infer.py", line 125, in main
    sess.run([])
  File "/mnt/c/work/2017/tf/env-linux/lib/python3.5/site-packages/tensorflow/python/training/monitored_session.py", line 462, in run
    run_metadata=run_metadata)
  File "/mnt/c/work/2017/tf/env-linux/lib/python3.5/site-packages/tensorflow/python/training/monitored_session.py", line 786, in run
    run_metadata=run_metadata)
  File "/mnt/c/work/2017/tf/env-linux/lib/python3.5/site-packages/tensorflow/python/training/monitored_session.py", line 744, in run
    return self._sess.run(*args, **kwargs)
  File "/mnt/c/work/2017/tf/env-linux/lib/python3.5/site-packages/tensorflow/python/training/monitored_session.py", line 899, in run
    run_metadata=run_metadata))
  File "/mnt/c/work/2017/tf/seq2seq/seq2seq/tasks/dump_attention.py", line 126, in after_run
    _create_figure(fetches)
  File "/mnt/c/work/2017/tf/seq2seq/seq2seq/tasks/dump_attention.py", line 58, in _create_figure
    fig = plt.figure(figsize=(8, 8))
  File "/mnt/c/work/2017/tf/env-linux/lib/python3.5/site-packages/matplotlib/pyplot.py", line 535, in figure
    **kwargs)
  File "/mnt/c/work/2017/tf/env-linux/lib/python3.5/site-packages/matplotlib/backends/backend_tkagg.py", line 81, in new_figure_manager
    return new_figure_manager_given_figure(num, figure)
  File "/mnt/c/work/2017/tf/env-linux/lib/python3.5/site-packages/matplotlib/backends/backend_tkagg.py", line 89, in new_figure_manager_given_figure
    window = Tk.Tk()
  File "/usr/local/lib/python3.5/tkinter/__init__.py", line 1877, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

----------------------------------------------------------------------
Ran 2 tests in 12.785s

FAILED (errors=1)
Segmentation fault (core dumped)

じゃあ vcXsrv を実行しておこうか。

$ export DISPLAY=localhost:0

やり直す。

INFO:tensorflow:Restored model from /tmp/tmp3j3759e5/model.ckpt-50
c
c
c
c
.
----------------------------------------------------------------------
Ran 2 tests in 19.904s

OK
*** Error in `python': double free or corruption (!prev): 0x00000000013d2da0 ***
Aborted (core dumped)

たぶん正常終了なのだが、ウィンドウは表示されず、 core dumped で終わった。

やっぱり Mac か Ubuntu でやるのが素直らしい。。

だが matplotlib にこだわらなければ Windows でも BoW でも動くものはある。

> python -m unittest seq2seq.test.data_test

などなど。。詳細の調査はこれから。

ja.nishimotz.com

目次

TensorFlow

Windows

tf-seq2seq

Bash on Ubuntu on Windows で試す