master
/ job_logs / job-gpu-62b950d6f9c7fbd55d4e9e0f.log

job-gpu-62b950d6f9c7fbd55d4e9e0f.log @533d585 raw · history · blame

2022-06-27T06:40:28.664029384Z SYSTEM: Preparing env...
2022-06-27T06:40:29.161948578Z SYSTEM: Running...
2022-06-27T06:40:29.90655155Z 2022-06-27 14:40:29.905643: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T06:40:31.552840979Z WARNING:tensorflow:From /home/jovyan/.virtualenvs/basenv/lib/python3.7/site-packages/tensorflow/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
2022-06-27T06:40:31.552884694Z Instructions for updating:
2022-06-27T06:40:31.552891704Z non-resource variables are not supported in the long term
2022-06-27T06:40:31.557248833Z Imported model (for Places365, 128x128 images)
2022-06-27T06:40:34.255714665Z WARNING:tensorflow:From /home/jovyan/work/src/model.py:20: conv2d (from tensorflow.python.keras.legacy_tf_layers.convolutional) is deprecated and will be removed in a future version.
2022-06-27T06:40:34.255762327Z Instructions for updating:
2022-06-27T06:40:34.255770327Z Use `tf.keras.layers.Conv2D` instead.
2022-06-27T06:40:34.255776823Z WARNING:tensorflow:From /home/jovyan/.virtualenvs/basenv/lib/python3.7/site-packages/tensorflow/python/keras/legacy_tf_layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer_v1) is deprecated and will be removed in a future version.
2022-06-27T06:40:34.255788806Z Instructions for updating:
2022-06-27T06:40:34.255793807Z Please use `layer.__call__` method instead.
2022-06-27T06:40:34.385566834Z WARNING:tensorflow:From /home/jovyan/work/src/model.py:79: conv2d_transpose (from tensorflow.python.keras.legacy_tf_layers.convolutional) is deprecated and will be removed in a future version.
2022-06-27T06:40:34.385615911Z Instructions for updating:
2022-06-27T06:40:34.385623287Z Use `tf.keras.layers.Conv2DTranspose` instead.
2022-06-27T06:40:34.46652205Z 2022-06-27 14:40:34.463569: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2022-06-27T06:40:34.587041885Z 2022-06-27 14:40:34.558264: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:34.587091586Z 2022-06-27 14:40:34.559496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
2022-06-27T06:40:34.587099226Z pciBusID: 0000:00:08.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
2022-06-27T06:40:34.587104431Z coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 7.75GiB deviceMemoryBandwidth: 681.88GiB/s
2022-06-27T06:40:34.587110049Z 2022-06-27 14:40:34.559547: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T06:40:34.587115546Z 2022-06-27 14:40:34.563201: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2022-06-27T06:40:34.587121023Z 2022-06-27 14:40:34.566815: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2022-06-27T06:40:34.587142859Z 2022-06-27 14:40:34.567434: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2022-06-27T06:40:34.587149143Z 2022-06-27 14:40:34.571508: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2022-06-27T06:40:34.587154254Z 2022-06-27 14:40:34.573557: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2022-06-27T06:40:34.587159273Z 2022-06-27 14:40:34.581702: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2022-06-27T06:40:34.58717697Z 2022-06-27 14:40:34.582161: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:34.587184192Z 2022-06-27 14:40:34.583629: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:34.587191825Z 2022-06-27 14:40:34.584731: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2022-06-27T06:40:34.587196874Z 2022-06-27 14:40:34.585721: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 FMA
2022-06-27T06:40:34.587202666Z To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-06-27T06:40:34.605273814Z 2022-06-27 14:40:34.604881: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2499990000 Hz
2022-06-27T06:40:34.612440387Z 2022-06-27 14:40:34.611455: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x43ef4a0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-06-27T06:40:34.612469486Z 2022-06-27 14:40:34.611536: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2022-06-27T06:40:34.818065383Z 2022-06-27 14:40:34.816854: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:34.818226758Z 2022-06-27 14:40:34.817945: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1133430 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-06-27T06:40:34.818242687Z 2022-06-27 14:40:34.818059: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla P100-PCIE-16GB, Compute Capability 6.0
2022-06-27T06:40:34.819098669Z 2022-06-27 14:40:34.818920: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:34.826000267Z 2022-06-27 14:40:34.820425: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
2022-06-27T06:40:34.826040277Z pciBusID: 0000:00:08.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
2022-06-27T06:40:34.826047919Z coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 7.75GiB deviceMemoryBandwidth: 681.88GiB/s
2022-06-27T06:40:34.82605344Z 2022-06-27 14:40:34.820483: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T06:40:34.82605896Z 2022-06-27 14:40:34.820538: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2022-06-27T06:40:34.826064758Z 2022-06-27 14:40:34.820589: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2022-06-27T06:40:34.826069913Z 2022-06-27 14:40:34.820637: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2022-06-27T06:40:34.826075151Z 2022-06-27 14:40:34.820683: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2022-06-27T06:40:34.826080279Z 2022-06-27 14:40:34.820757: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2022-06-27T06:40:34.826085367Z 2022-06-27 14:40:34.820807: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2022-06-27T06:40:34.826090711Z 2022-06-27 14:40:34.821073: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:34.8260964Z 2022-06-27 14:40:34.822732: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:34.826101603Z 2022-06-27 14:40:34.824064: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2022-06-27T06:40:34.826106851Z 2022-06-27 14:40:34.824140: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2022-06-27T06:40:35.48051732Z 2022-06-27 14:40:35.478735: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-06-27T06:40:35.480572834Z 2022-06-27 14:40:35.478800: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2022-06-27T06:40:35.480580695Z 2022-06-27 14:40:35.478813: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2022-06-27T06:40:35.491016804Z 2022-06-27 14:40:35.488592: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:35.492925518Z 2022-06-27 14:40:35.491127: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27T06:40:35.492963862Z 2022-06-27 14:40:35.492080: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7116 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:08.0, compute capability: 6.0)
2022-06-27T06:40:35.830526984Z 2022-06-27 14:40:35.825663: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2022-06-27T06:40:37.078582913Z 2022-06-27 14:40:37.071379: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2022-06-27T06:40:39.337113298Z SYSTEM: Finishing...