master
/ job_logs / job-gpu-62b96ec3f901e7972521f6bc.log

job-gpu-62b96ec3f901e7972521f6bc.log @533d585 — raw · history · blame 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
2022-06-27T08:48:09.24239905Z SYSTEM: Preparing env...
2022-06-27T08:48:09.786941439Z SYSTEM: Running...
2022-06-27T08:48:10.51858763Z 2022-06-27 16:48:10.517635: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T08:48:12.178056383Z WARNING:tensorflow:From /home/jovyan/.virtualenvs/basenv/lib/python3.7/site-packages/tensorflow/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
2022-06-27T08:48:12.178109572Z Instructions for updating:
2022-06-27T08:48:12.178117458Z non-resource variables are not supported in the long term
2022-06-27T08:48:12.183152171Z Imported model (for Places365, 128x128 images)
2022-06-27T08:48:14.874564815Z WARNING:tensorflow:From /home/jovyan/work/src/model.py:20: conv2d (from tensorflow.python.keras.legacy_tf_layers.convolutional) is deprecated and will be removed in a future version.
2022-06-27T08:48:14.87460536Z Instructions for updating:
2022-06-27T08:48:14.874612991Z Use `tf.keras.layers.Conv2D` instead.
2022-06-27T08:48:14.879516836Z WARNING:tensorflow:From /home/jovyan/.virtualenvs/basenv/lib/python3.7/site-packages/tensorflow/python/keras/legacy_tf_layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer_v1) is deprecated and will be removed in a future version.
2022-06-27T08:48:14.879552898Z Instructions for updating:
2022-06-27T08:48:14.879556966Z Please use `layer.__call__` method instead.
2022-06-27T08:48:14.998783159Z WARNING:tensorflow:From /home/jovyan/work/src/model.py:79: conv2d_transpose (from tensorflow.python.keras.legacy_tf_layers.convolutional) is deprecated and will be removed in a future version.
2022-06-27T08:48:14.99882407Z Instructions for updating:
2022-06-27T08:48:14.998827896Z Use `tf.keras.layers.Conv2DTranspose` instead.
2022-06-27T08:48:15.082132374Z 2022-06-27 16:48:15.075591: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2022-06-27T08:48:15.167539823Z 2022-06-27 16:48:15.154770: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:15.167590578Z 2022-06-27 16:48:15.156152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
2022-06-27T08:48:15.16759558Z pciBusID: 0000:00:08.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
2022-06-27T08:48:15.167598782Z coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 7.75GiB deviceMemoryBandwidth: 681.88GiB/s
2022-06-27T08:48:15.167602013Z 2022-06-27 16:48:15.156213: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T08:48:15.167605072Z 2022-06-27 16:48:15.160218: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2022-06-27T08:48:15.167608115Z 2022-06-27 16:48:15.163802: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2022-06-27T08:48:15.167628968Z 2022-06-27 16:48:15.164231: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2022-06-27T08:48:15.172464872Z 2022-06-27 16:48:15.167436: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2022-06-27T08:48:15.17248651Z 2022-06-27 16:48:15.168987: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2022-06-27T08:48:15.179400732Z 2022-06-27 16:48:15.175372: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2022-06-27T08:48:15.179421972Z 2022-06-27 16:48:15.175712: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:15.179428301Z 2022-06-27 16:48:15.176844: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:15.17943329Z 2022-06-27 16:48:15.177689: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2022-06-27T08:48:15.179438827Z 2022-06-27 16:48:15.178507: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 FMA
2022-06-27T08:48:15.179443601Z To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-06-27T08:48:15.189791017Z 2022-06-27 16:48:15.189490: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2499990000 Hz
2022-06-27T08:48:15.190826889Z 2022-06-27 16:48:15.190353: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5d4d3b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-06-27T08:48:15.190840436Z 2022-06-27 16:48:15.190405: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2022-06-27T08:48:15.354103229Z 2022-06-27 16:48:15.352890: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:15.360495813Z 2022-06-27 16:48:15.353962: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2a91430 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-06-27T08:48:15.36051428Z 2022-06-27 16:48:15.354039: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla P100-PCIE-16GB, Compute Capability 6.0
2022-06-27T08:48:15.360518449Z 2022-06-27 16:48:15.354703: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:15.360521962Z 2022-06-27 16:48:15.355674: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
2022-06-27T08:48:15.360540021Z pciBusID: 0000:00:08.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
2022-06-27T08:48:15.360543183Z coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 7.75GiB deviceMemoryBandwidth: 681.88GiB/s
2022-06-27T08:48:15.360546053Z 2022-06-27 16:48:15.355713: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T08:48:15.360548859Z 2022-06-27 16:48:15.355757: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2022-06-27T08:48:15.360551628Z 2022-06-27 16:48:15.355789: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2022-06-27T08:48:15.360554374Z 2022-06-27 16:48:15.355815: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2022-06-27T08:48:15.360557367Z 2022-06-27 16:48:15.355840: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2022-06-27T08:48:15.360560106Z 2022-06-27 16:48:15.355863: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2022-06-27T08:48:15.360562889Z 2022-06-27 16:48:15.355889: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2022-06-27T08:48:15.360566266Z 2022-06-27 16:48:15.356094: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:15.360569337Z 2022-06-27 16:48:15.357167: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:15.360572245Z 2022-06-27 16:48:15.358197: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2022-06-27T08:48:15.360574967Z 2022-06-27 16:48:15.358256: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T08:48:16.025531234Z 2022-06-27 16:48:16.024309: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-06-27T08:48:16.02556822Z 2022-06-27 16:48:16.024381: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2022-06-27T08:48:16.02557269Z 2022-06-27 16:48:16.024394: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2022-06-27T08:48:16.036018785Z 2022-06-27 16:48:16.033186: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:16.036045308Z 2022-06-27 16:48:16.034241: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T08:48:16.036052087Z 2022-06-27 16:48:16.035136: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7116 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:08.0, compute capability: 6.0)
2022-06-27T08:48:16.324801304Z 2022-06-27 16:48:16.323386: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2022-06-27T08:48:17.556743331Z 2022-06-27 16:48:17.551710: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2022-06-27T08:48:19.783798978Z SYSTEM: Finishing...