-- Logs begin at Fri 2021-08-06 13:37:44 UTC, end at Fri 2021-08-06 16:36:12 UTC. -- Aug 06 14:09:21.281089 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 14:09:21.289382 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 14:09:21 2021] *** Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: nodename: dv-cinder-803748-1-iscsi Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: machine: x86_64 Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: clock source: unix Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: pcre jit disabled Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: detected number of CPU cores: 4 Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: current working directory: / Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: detected binary path: /usr/bin/uwsgi-core Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: your processes number limit is 47680 Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: your memory page size is 4096 bytes Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: detected max file descriptor number: 2048 Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: lock engine: pthread robust mutexes Aug 06 14:09:21.292840 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: thunder lock: enabled Aug 06 14:09:21.293731 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 14:09:21.293731 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 14:09:21.337111 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: Python main interpreter initialized at 0x55cc066da9c0 Aug 06 14:09:21.337111 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: python threads support enabled Aug 06 14:09:21.337111 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: your server socket listen backlog is limited to 100 connections Aug 06 14:09:21.337111 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: your mercy for graceful operations on workers is 90 seconds Aug 06 14:09:21.337509 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 14:09:21.337585 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: *** Operational MODE: preforking *** Aug 06 14:09:21.337812 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: *** uWSGI is running in multiple interpreter mode *** Aug 06 14:09:21.337812 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: spawned uWSGI master process (pid: 100255) Aug 06 14:09:21.338020 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 14:09:21.338808 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: spawned uWSGI worker 1 (pid: 100260, cores: 1) Aug 06 14:09:21.338973 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: spawned uWSGI worker 2 (pid: 100261, cores: 1) Aug 06 14:09:21.339132 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 14:09:22.638731 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: Traceback (most recent call last): Aug 06 14:09:22.638731 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: from cinder.common import config Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: from cinder import policies Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: from cinder.policies import attachments Aug 06 14:09:22.639404 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:09:22.640092 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: from cinder.policies import base Aug 06 14:09:22.640092 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:09:22.640092 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:09:22.640092 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:09:22.640092 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:09:22.640092 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: *** no app loaded. going in full dynamic mode *** Aug 06 14:09:22.640092 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: --- no python application found, check your startup logs for errors --- Aug 06 14:09:22.642598 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: [pid: 100260|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 14:09:22 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:09:22.656398 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: Traceback (most recent call last): Aug 06 14:09:22.656398 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:09:22.656997 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:09:22.656997 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:09:22.656997 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: from cinder.common import config Aug 06 14:09:22.656997 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:09:22.656997 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:09:22.656997 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:09:22.658120 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: from cinder import policies Aug 06 14:09:22.658120 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:09:22.658266 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: from cinder.policies import attachments Aug 06 14:09:22.658266 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:09:22.658532 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: from cinder.policies import base Aug 06 14:09:22.658532 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:09:22.658802 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:09:22.658802 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:09:22.658960 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:09:22.658960 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: *** no app loaded. going in full dynamic mode *** Aug 06 14:09:26.440554 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: --- no python application found, check your startup logs for errors --- Aug 06 14:09:26.440554 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: [pid: 100260|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 14:09:26 2021] POST /volume/v3/5234c8bede28489d96854e423b8f2b92/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:10:11.474506 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: Fri Aug 6 14:10:11 2021 - graceful shutdown triggered... Aug 06 14:10:11.474973 dv-cinder-803748-1-iscsi devstack@c-api.service[100260]: Gracefully killing worker 1 (pid: 100260)... Aug 06 14:10:11.474638 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 14:10:11.475460 dv-cinder-803748-1-iscsi devstack@c-api.service[100261]: Gracefully killing worker 2 (pid: 100261)... Aug 06 14:10:12.475624 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: worker 1 buried after 1 seconds Aug 06 14:10:12.476027 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: worker 2 buried after 1 seconds Aug 06 14:10:12.476027 dv-cinder-803748-1-iscsi devstack@c-api.service[100255]: goodbye to uWSGI. Aug 06 14:10:12.477143 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 14:10:12.477678 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 14:21:26.711182 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 14:21:26.719101 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 14:21:26.721082 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 14:21:26 2021] *** Aug 06 14:21:26.721082 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 14:21:26.721082 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 14:21:26.721082 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: nodename: dv-cinder-803748-1-iscsi Aug 06 14:21:26.721082 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: machine: x86_64 Aug 06 14:21:26.721082 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: clock source: unix Aug 06 14:21:26.721082 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: pcre jit disabled Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: detected number of CPU cores: 4 Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: current working directory: / Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: detected binary path: /usr/bin/uwsgi-core Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: your processes number limit is 47680 Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: your memory page size is 4096 bytes Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: detected max file descriptor number: 2048 Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: lock engine: pthread robust mutexes Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: thunder lock: enabled Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 14:21:26.721581 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 14:21:26.768745 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: Python main interpreter initialized at 0x56352e49f9c0 Aug 06 14:21:26.768745 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: python threads support enabled Aug 06 14:21:26.768745 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: your server socket listen backlog is limited to 100 connections Aug 06 14:21:26.768745 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: your mercy for graceful operations on workers is 90 seconds Aug 06 14:21:26.769166 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 14:21:26.769243 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: *** Operational MODE: preforking *** Aug 06 14:21:26.769479 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: *** uWSGI is running in multiple interpreter mode *** Aug 06 14:21:26.769479 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: spawned uWSGI master process (pid: 160254) Aug 06 14:21:26.769690 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 14:21:26.770523 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: spawned uWSGI worker 1 (pid: 160258, cores: 1) Aug 06 14:21:26.770631 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: spawned uWSGI worker 2 (pid: 160259, cores: 1) Aug 06 14:21:26.770786 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 14:21:28.091210 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: Traceback (most recent call last): Aug 06 14:21:28.091210 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: from cinder.common import config Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: from cinder import policies Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: from cinder.policies import attachments Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: from cinder.policies import base Aug 06 14:21:28.092072 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:21:28.092761 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:21:28.092761 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:21:28.092761 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:21:28.092761 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: *** no app loaded. going in full dynamic mode *** Aug 06 14:21:28.092761 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: --- no python application found, check your startup logs for errors --- Aug 06 14:21:28.092761 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: [pid: 160258|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 14:21:28 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:21:28.095364 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: Traceback (most recent call last): Aug 06 14:21:28.095364 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:21:28.095891 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:21:28.095891 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:21:28.095891 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: from cinder.common import config Aug 06 14:21:28.095891 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:21:28.095891 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:21:28.095891 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:21:28.095891 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: from cinder import policies Aug 06 14:21:28.096256 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:21:28.096256 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: from cinder.policies import attachments Aug 06 14:21:28.096256 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:21:28.096256 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: from cinder.policies import base Aug 06 14:21:28.096256 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:21:28.096256 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:21:28.096256 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:21:28.096605 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:21:28.096605 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: *** no app loaded. going in full dynamic mode *** Aug 06 14:21:31.943201 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: --- no python application found, check your startup logs for errors --- Aug 06 14:21:31.943201 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: [pid: 160258|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 14:21:31 2021] POST /volume/v3/da93d573fd4f4e6889c06024790b93ec/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:22:17.208531 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: Fri Aug 6 14:22:17 2021 - graceful shutdown triggered... Aug 06 14:22:17.210526 dv-cinder-803748-1-iscsi devstack@c-api.service[160258]: Gracefully killing worker 1 (pid: 160258)... Aug 06 14:22:17.209411 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 14:22:17.211071 dv-cinder-803748-1-iscsi devstack@c-api.service[160259]: Gracefully killing worker 2 (pid: 160259)... Aug 06 14:22:18.209747 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: worker 1 buried after 1 seconds Aug 06 14:22:18.209747 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: worker 2 buried after 1 seconds Aug 06 14:22:18.209747 dv-cinder-803748-1-iscsi devstack@c-api.service[160254]: goodbye to uWSGI. Aug 06 14:22:18.211784 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 14:22:18.212332 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 14:33:50.711275 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 14:33:50.724987 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 14:33:50 2021] *** Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: nodename: dv-cinder-803748-1-iscsi Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: machine: x86_64 Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: clock source: unix Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: pcre jit disabled Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: detected number of CPU cores: 4 Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: current working directory: / Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: detected binary path: /usr/bin/uwsgi-core Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: your processes number limit is 47680 Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: your memory page size is 4096 bytes Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: detected max file descriptor number: 2048 Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: lock engine: pthread robust mutexes Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: thunder lock: enabled Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 14:33:50.730178 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 14:33:50.784397 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: Python main interpreter initialized at 0x55deed3cf9c0 Aug 06 14:33:50.784397 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: python threads support enabled Aug 06 14:33:50.784397 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: your server socket listen backlog is limited to 100 connections Aug 06 14:33:50.784397 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: your mercy for graceful operations on workers is 90 seconds Aug 06 14:33:50.785130 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 14:33:50.785130 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: *** Operational MODE: preforking *** Aug 06 14:33:50.785130 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: *** uWSGI is running in multiple interpreter mode *** Aug 06 14:33:50.785130 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: spawned uWSGI master process (pid: 227051) Aug 06 14:33:50.785497 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 14:33:50.795205 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: spawned uWSGI worker 1 (pid: 227052, cores: 1) Aug 06 14:33:50.795205 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: spawned uWSGI worker 2 (pid: 227053, cores: 1) Aug 06 14:33:50.795205 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 14:33:52.196224 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: Traceback (most recent call last): Aug 06 14:33:52.196224 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: from cinder.common import config Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: from cinder import policies Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: from cinder.policies import attachments Aug 06 14:33:52.197125 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: from cinder.policies import base Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: *** no app loaded. going in full dynamic mode *** Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: --- no python application found, check your startup logs for errors --- Aug 06 14:33:52.197705 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: [pid: 227052|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 14:33:52 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:33:52.385730 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: Traceback (most recent call last): Aug 06 14:33:52.385730 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:33:52.385730 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:33:52.385730 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: from cinder.common import config Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: from cinder import policies Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: from cinder.policies import attachments Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: from cinder.policies import base Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:33:52.386529 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: *** no app loaded. going in full dynamic mode *** Aug 06 14:33:56.424971 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: --- no python application found, check your startup logs for errors --- Aug 06 14:33:56.424971 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: [pid: 227052|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 14:33:56 2021] POST /volume/v3/bca888745af644f094619875f41ebfd6/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:34:41.088317 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: Fri Aug 6 14:34:41 2021 - graceful shutdown triggered... Aug 06 14:34:41.088850 dv-cinder-803748-1-iscsi devstack@c-api.service[227052]: Gracefully killing worker 1 (pid: 227052)... Aug 06 14:34:41.088442 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 14:34:41.089437 dv-cinder-803748-1-iscsi devstack@c-api.service[227053]: Gracefully killing worker 2 (pid: 227053)... Aug 06 14:34:42.089238 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: worker 1 buried after 1 seconds Aug 06 14:34:42.089238 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: worker 2 buried after 1 seconds Aug 06 14:34:42.089238 dv-cinder-803748-1-iscsi devstack@c-api.service[227051]: goodbye to uWSGI. Aug 06 14:34:42.090716 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 14:34:42.091797 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 14:46:18.054422 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 14:46:18.061756 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 14:46:18 2021] *** Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: nodename: dv-cinder-803748-1-iscsi Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: machine: x86_64 Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: clock source: unix Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: pcre jit disabled Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: detected number of CPU cores: 4 Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: current working directory: / Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: detected binary path: /usr/bin/uwsgi-core Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: your processes number limit is 47680 Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: your memory page size is 4096 bytes Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: detected max file descriptor number: 2048 Aug 06 14:46:18.066983 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: lock engine: pthread robust mutexes Aug 06 14:46:18.067966 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: thunder lock: enabled Aug 06 14:46:18.068274 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 14:46:18.068408 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 14:46:18.113769 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: Python main interpreter initialized at 0x55b3f092f9c0 Aug 06 14:46:18.113769 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: python threads support enabled Aug 06 14:46:18.113769 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: your server socket listen backlog is limited to 100 connections Aug 06 14:46:18.113769 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: your mercy for graceful operations on workers is 90 seconds Aug 06 14:46:18.114290 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 14:46:18.114290 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: *** Operational MODE: preforking *** Aug 06 14:46:18.114495 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: *** uWSGI is running in multiple interpreter mode *** Aug 06 14:46:18.114495 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: spawned uWSGI master process (pid: 287099) Aug 06 14:46:18.114702 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 14:46:18.117115 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: spawned uWSGI worker 1 (pid: 287114, cores: 1) Aug 06 14:46:18.117115 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: spawned uWSGI worker 2 (pid: 287115, cores: 1) Aug 06 14:46:18.117115 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 14:46:19.445342 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: Traceback (most recent call last): Aug 06 14:46:19.445342 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: from cinder.common import config Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: from cinder import policies Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: from cinder.policies import attachments Aug 06 14:46:19.446102 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: from cinder.policies import base Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: *** no app loaded. going in full dynamic mode *** Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: --- no python application found, check your startup logs for errors --- Aug 06 14:46:19.448772 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: [pid: 287115|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 14:46:19 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:46:19.451122 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: Traceback (most recent call last): Aug 06 14:46:19.451122 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 14:46:19.451653 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: from cinder.wsgi.wsgi import initialize_application Aug 06 14:46:19.451653 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 14:46:19.451653 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: from cinder.common import config Aug 06 14:46:19.451653 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 14:46:19.451653 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 14:46:19.451653 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 14:46:19.451653 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: from cinder import policies Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: from cinder.policies import attachments Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: from cinder.policies import base Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 14:46:19.452090 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: *** no app loaded. going in full dynamic mode *** Aug 06 14:46:23.294668 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: --- no python application found, check your startup logs for errors --- Aug 06 14:46:23.294668 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: [pid: 287115|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 14:46:23 2021] POST /volume/v3/34050d99b48e474287088e829a5de4bf/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 14:47:08.495917 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: Fri Aug 6 14:47:08 2021 - graceful shutdown triggered... Aug 06 14:47:08.496450 dv-cinder-803748-1-iscsi devstack@c-api.service[287114]: Gracefully killing worker 1 (pid: 287114)... Aug 06 14:47:08.496655 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 14:47:08.497029 dv-cinder-803748-1-iscsi devstack@c-api.service[287115]: Gracefully killing worker 2 (pid: 287115)... Aug 06 14:47:09.497061 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: worker 1 buried after 1 seconds Aug 06 14:47:09.497061 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: worker 2 buried after 1 seconds Aug 06 14:47:09.497061 dv-cinder-803748-1-iscsi devstack@c-api.service[287099]: goodbye to uWSGI. Aug 06 14:47:09.498628 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 14:47:09.499166 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 15:02:06.022159 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 15:02:06.029757 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 15:02:06 2021] *** Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: nodename: dv-cinder-803748-1-iscsi Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: machine: x86_64 Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: clock source: unix Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: pcre jit disabled Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: detected number of CPU cores: 4 Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: current working directory: / Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: detected binary path: /usr/bin/uwsgi-core Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: your processes number limit is 47680 Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: your memory page size is 4096 bytes Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: detected max file descriptor number: 2048 Aug 06 15:02:06.033365 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: lock engine: pthread robust mutexes Aug 06 15:02:06.034360 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: thunder lock: enabled Aug 06 15:02:06.034360 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 15:02:06.034360 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 15:02:06.082007 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: Python main interpreter initialized at 0x55acd86739c0 Aug 06 15:02:06.082007 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: python threads support enabled Aug 06 15:02:06.082007 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: your server socket listen backlog is limited to 100 connections Aug 06 15:02:06.082007 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: your mercy for graceful operations on workers is 90 seconds Aug 06 15:02:06.082529 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 15:02:06.082529 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: *** Operational MODE: preforking *** Aug 06 15:02:06.082783 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: *** uWSGI is running in multiple interpreter mode *** Aug 06 15:02:06.082783 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: spawned uWSGI master process (pid: 353085) Aug 06 15:02:06.082998 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 15:02:06.085452 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: spawned uWSGI worker 1 (pid: 353099, cores: 1) Aug 06 15:02:06.085452 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: spawned uWSGI worker 2 (pid: 353100, cores: 1) Aug 06 15:02:06.085452 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 15:02:07.419677 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: Traceback (most recent call last): Aug 06 15:02:07.419677 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: from cinder.common import config Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: from cinder import policies Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: from cinder.policies import attachments Aug 06 15:02:07.420393 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: from cinder.policies import base Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: *** no app loaded. going in full dynamic mode *** Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: --- no python application found, check your startup logs for errors --- Aug 06 15:02:07.424094 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: [pid: 353100|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 15:02:07 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:02:07.427538 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: Traceback (most recent call last): Aug 06 15:02:07.427538 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: from cinder.common import config Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: from cinder import policies Aug 06 15:02:07.428136 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: from cinder.policies import attachments Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: from cinder.policies import base Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:02:07.428734 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: *** no app loaded. going in full dynamic mode *** Aug 06 15:02:11.343132 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: --- no python application found, check your startup logs for errors --- Aug 06 15:02:11.343132 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: [pid: 353100|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 15:02:11 2021] POST /volume/v3/6684e0725b1f43a68a9a9564bcc8b63a/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:02:56.337440 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: Fri Aug 6 15:02:56 2021 - graceful shutdown triggered... Aug 06 15:02:56.337942 dv-cinder-803748-1-iscsi devstack@c-api.service[353100]: Gracefully killing worker 2 (pid: 353100)... Aug 06 15:02:56.338403 dv-cinder-803748-1-iscsi devstack@c-api.service[353099]: Gracefully killing worker 1 (pid: 353099)... Aug 06 15:02:56.339189 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 15:02:57.338575 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: worker 1 buried after 1 seconds Aug 06 15:02:57.338575 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: worker 2 buried after 1 seconds Aug 06 15:02:57.338575 dv-cinder-803748-1-iscsi devstack@c-api.service[353085]: goodbye to uWSGI. Aug 06 15:02:57.340110 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 15:02:57.340646 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 15:14:21.201140 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 15:14:21.210444 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 15:14:21 2021] *** Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: nodename: dv-cinder-803748-1-iscsi Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: machine: x86_64 Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: clock source: unix Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: pcre jit disabled Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: detected number of CPU cores: 4 Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: current working directory: / Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: detected binary path: /usr/bin/uwsgi-core Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: your processes number limit is 47680 Aug 06 15:14:21.212435 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: your memory page size is 4096 bytes Aug 06 15:14:21.213143 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: detected max file descriptor number: 2048 Aug 06 15:14:21.213143 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: lock engine: pthread robust mutexes Aug 06 15:14:21.213143 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: thunder lock: enabled Aug 06 15:14:21.213143 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 15:14:21.213143 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 15:14:21.257583 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: Python main interpreter initialized at 0x5639ad4379c0 Aug 06 15:14:21.257583 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: python threads support enabled Aug 06 15:14:21.257583 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: your server socket listen backlog is limited to 100 connections Aug 06 15:14:21.257583 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: your mercy for graceful operations on workers is 90 seconds Aug 06 15:14:21.257930 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 15:14:21.257996 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: *** Operational MODE: preforking *** Aug 06 15:14:21.258307 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: *** uWSGI is running in multiple interpreter mode *** Aug 06 15:14:21.258307 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: spawned uWSGI master process (pid: 413107) Aug 06 15:14:21.258523 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 15:14:21.259417 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: spawned uWSGI worker 1 (pid: 413108, cores: 1) Aug 06 15:14:21.259514 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: spawned uWSGI worker 2 (pid: 413109, cores: 1) Aug 06 15:14:21.259651 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 15:14:22.560260 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: Traceback (most recent call last): Aug 06 15:14:22.560260 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: from cinder.common import config Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: from cinder import policies Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: from cinder.policies import attachments Aug 06 15:14:22.560902 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: from cinder.policies import base Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: *** no app loaded. going in full dynamic mode *** Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: --- no python application found, check your startup logs for errors --- Aug 06 15:14:22.561465 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: [pid: 413109|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 15:14:22 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:14:22.583385 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: Traceback (most recent call last): Aug 06 15:14:22.583385 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: from cinder.common import config Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: from cinder import policies Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:14:22.583989 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: from cinder.policies import attachments Aug 06 15:14:22.585029 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:14:22.585029 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: from cinder.policies import base Aug 06 15:14:22.585029 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:14:22.585029 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:14:22.585029 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:14:22.585029 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:14:22.585029 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: *** no app loaded. going in full dynamic mode *** Aug 06 15:14:26.441789 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: --- no python application found, check your startup logs for errors --- Aug 06 15:14:26.441789 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: [pid: 413109|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 15:14:26 2021] POST /volume/v3/00821b6d4cd14776ac06bec24d211eac/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:15:11.720589 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 15:15:11.721345 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: Fri Aug 6 15:15:11 2021 - graceful shutdown triggered... Aug 06 15:15:11.721813 dv-cinder-803748-1-iscsi devstack@c-api.service[413109]: Gracefully killing worker 2 (pid: 413109)... Aug 06 15:15:11.724453 dv-cinder-803748-1-iscsi devstack@c-api.service[413108]: Gracefully killing worker 1 (pid: 413108)... Aug 06 15:15:12.722599 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: worker 1 buried after 1 seconds Aug 06 15:15:12.722599 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: worker 2 buried after 1 seconds Aug 06 15:15:12.722599 dv-cinder-803748-1-iscsi devstack@c-api.service[413107]: goodbye to uWSGI. Aug 06 15:15:12.724024 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 15:15:12.724572 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 15:26:33.429449 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 15:26:33.437659 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 15:26:33.440497 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 15:26:33 2021] *** Aug 06 15:26:33.440664 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 15:26:33.440785 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 15:26:33.440882 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: nodename: dv-cinder-803748-1-iscsi Aug 06 15:26:33.440993 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: machine: x86_64 Aug 06 15:26:33.441106 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: clock source: unix Aug 06 15:26:33.441228 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: pcre jit disabled Aug 06 15:26:33.441344 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: detected number of CPU cores: 4 Aug 06 15:26:33.441453 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: current working directory: / Aug 06 15:26:33.441682 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: detected binary path: /usr/bin/uwsgi-core Aug 06 15:26:33.441809 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: your processes number limit is 47680 Aug 06 15:26:33.441921 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: your memory page size is 4096 bytes Aug 06 15:26:33.442062 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: detected max file descriptor number: 2048 Aug 06 15:26:33.442197 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: lock engine: pthread robust mutexes Aug 06 15:26:33.442472 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: thunder lock: enabled Aug 06 15:26:33.442770 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 15:26:33.442960 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 15:26:33.487991 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: Python main interpreter initialized at 0x55eab4e8d9d0 Aug 06 15:26:33.487991 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: python threads support enabled Aug 06 15:26:33.487991 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: your server socket listen backlog is limited to 100 connections Aug 06 15:26:33.487991 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: your mercy for graceful operations on workers is 90 seconds Aug 06 15:26:33.488314 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 15:26:33.488403 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: *** Operational MODE: preforking *** Aug 06 15:26:33.488685 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: *** uWSGI is running in multiple interpreter mode *** Aug 06 15:26:33.488685 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: spawned uWSGI master process (pid: 473518) Aug 06 15:26:33.488892 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 15:26:33.489468 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: spawned uWSGI worker 1 (pid: 473519, cores: 1) Aug 06 15:26:33.489837 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: spawned uWSGI worker 2 (pid: 473520, cores: 1) Aug 06 15:26:33.490070 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 15:26:34.840724 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: Traceback (most recent call last): Aug 06 15:26:34.840724 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: from cinder.common import config Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: from cinder import policies Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: from cinder.policies import attachments Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: from cinder.policies import base Aug 06 15:26:34.841410 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:26:34.844338 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:26:34.844338 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:26:34.844338 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:26:34.844338 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: *** no app loaded. going in full dynamic mode *** Aug 06 15:26:34.844338 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: --- no python application found, check your startup logs for errors --- Aug 06 15:26:34.844338 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: [pid: 473520|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 15:26:34 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:26:34.875638 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: Traceback (most recent call last): Aug 06 15:26:34.875638 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: from cinder.common import config Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: from cinder import policies Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: from cinder.policies import attachments Aug 06 15:26:34.876341 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:26:34.876906 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: from cinder.policies import base Aug 06 15:26:34.876906 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:26:34.876906 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:26:34.876906 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:26:34.876906 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:26:34.876906 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: *** no app loaded. going in full dynamic mode *** Aug 06 15:26:38.774635 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: --- no python application found, check your startup logs for errors --- Aug 06 15:26:38.774635 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: [pid: 473520|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 15:26:38 2021] POST /volume/v3/5b21edd8432a4406b1258aa2c59e6229/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:27:24.080688 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 15:27:24.081030 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: Fri Aug 6 15:27:24 2021 - graceful shutdown triggered... Aug 06 15:27:24.081496 dv-cinder-803748-1-iscsi devstack@c-api.service[473520]: Gracefully killing worker 2 (pid: 473520)... Aug 06 15:27:24.081864 dv-cinder-803748-1-iscsi devstack@c-api.service[473519]: Gracefully killing worker 1 (pid: 473519)... Aug 06 15:27:25.082138 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: worker 1 buried after 1 seconds Aug 06 15:27:25.082138 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: worker 2 buried after 1 seconds Aug 06 15:27:25.082697 dv-cinder-803748-1-iscsi devstack@c-api.service[473518]: goodbye to uWSGI. Aug 06 15:27:25.083658 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 15:27:25.084203 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 15:38:40.088368 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 15:38:40.096400 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 15:38:40 2021] *** Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: nodename: dv-cinder-803748-1-iscsi Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: machine: x86_64 Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: clock source: unix Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: pcre jit disabled Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: detected number of CPU cores: 4 Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: current working directory: / Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: detected binary path: /usr/bin/uwsgi-core Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: your processes number limit is 47680 Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: your memory page size is 4096 bytes Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: detected max file descriptor number: 2048 Aug 06 15:38:40.099579 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: lock engine: pthread robust mutexes Aug 06 15:38:40.100497 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: thunder lock: enabled Aug 06 15:38:40.100497 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 15:38:40.100497 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 15:38:40.143581 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: Python main interpreter initialized at 0x556cccb139d0 Aug 06 15:38:40.143581 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: python threads support enabled Aug 06 15:38:40.143581 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: your server socket listen backlog is limited to 100 connections Aug 06 15:38:40.143581 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: your mercy for graceful operations on workers is 90 seconds Aug 06 15:38:40.144043 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 15:38:40.144043 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: *** Operational MODE: preforking *** Aug 06 15:38:40.144290 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: *** uWSGI is running in multiple interpreter mode *** Aug 06 15:38:40.144290 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: spawned uWSGI master process (pid: 533622) Aug 06 15:38:40.144505 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 15:38:40.147612 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: spawned uWSGI worker 1 (pid: 533627, cores: 1) Aug 06 15:38:40.147612 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: spawned uWSGI worker 2 (pid: 533628, cores: 1) Aug 06 15:38:40.147612 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 15:38:41.451772 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: Traceback (most recent call last): Aug 06 15:38:41.451772 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: from cinder.common import config Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: from cinder import policies Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: from cinder.policies import attachments Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: from cinder.policies import base Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:38:41.452682 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: *** no app loaded. going in full dynamic mode *** Aug 06 15:38:41.453702 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: --- no python application found, check your startup logs for errors --- Aug 06 15:38:41.453702 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: [pid: 533628|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 15:38:41 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:38:41.460088 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: Traceback (most recent call last): Aug 06 15:38:41.460088 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: from cinder.common import config Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: from cinder import policies Aug 06 15:38:41.460651 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: from cinder.policies import attachments Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: from cinder.policies import base Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:38:41.461437 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: *** no app loaded. going in full dynamic mode *** Aug 06 15:38:45.301922 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: --- no python application found, check your startup logs for errors --- Aug 06 15:38:45.302502 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: [pid: 533628|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 15:38:45 2021] POST /volume/v3/46dced05196746bbbf594111fd80131e/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:39:30.269456 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: Fri Aug 6 15:39:30 2021 - graceful shutdown triggered... Aug 06 15:39:30.269985 dv-cinder-803748-1-iscsi devstack@c-api.service[533627]: Gracefully killing worker 1 (pid: 533627)... Aug 06 15:39:30.269578 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 15:39:30.270571 dv-cinder-803748-1-iscsi devstack@c-api.service[533628]: Gracefully killing worker 2 (pid: 533628)... Aug 06 15:39:31.270570 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: worker 1 buried after 1 seconds Aug 06 15:39:31.270989 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: worker 2 buried after 1 seconds Aug 06 15:39:31.270989 dv-cinder-803748-1-iscsi devstack@c-api.service[533622]: goodbye to uWSGI. Aug 06 15:39:31.272097 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 15:39:31.272641 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 15:54:23.548674 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 15:54:23.556626 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 15:54:23 2021] *** Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: nodename: dv-cinder-803748-1-iscsi Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: machine: x86_64 Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: clock source: unix Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: pcre jit disabled Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: detected number of CPU cores: 4 Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: current working directory: / Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: detected binary path: /usr/bin/uwsgi-core Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: your processes number limit is 47680 Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: your memory page size is 4096 bytes Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: detected max file descriptor number: 2048 Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: lock engine: pthread robust mutexes Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: thunder lock: enabled Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 15:54:23.561364 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 15:54:23.604148 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: Python main interpreter initialized at 0x562746a7e9d0 Aug 06 15:54:23.604148 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: python threads support enabled Aug 06 15:54:23.604148 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: your server socket listen backlog is limited to 100 connections Aug 06 15:54:23.604148 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: your mercy for graceful operations on workers is 90 seconds Aug 06 15:54:23.604614 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 15:54:23.604614 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: *** Operational MODE: preforking *** Aug 06 15:54:23.604869 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: *** uWSGI is running in multiple interpreter mode *** Aug 06 15:54:23.604869 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: spawned uWSGI master process (pid: 599305) Aug 06 15:54:23.605077 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 15:54:23.605685 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: spawned uWSGI worker 1 (pid: 599310, cores: 1) Aug 06 15:54:23.606060 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: spawned uWSGI worker 2 (pid: 599311, cores: 1) Aug 06 15:54:23.606200 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 15:54:24.953381 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: Traceback (most recent call last): Aug 06 15:54:24.953381 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: from cinder.common import config Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: from cinder import policies Aug 06 15:54:24.954093 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:54:24.954857 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: from cinder.policies import attachments Aug 06 15:54:24.954857 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:54:24.955142 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: from cinder.policies import base Aug 06 15:54:24.955142 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:54:24.955440 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:54:24.955541 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:54:24.955541 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:54:24.955717 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: *** no app loaded. going in full dynamic mode *** Aug 06 15:54:24.956116 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: --- no python application found, check your startup logs for errors --- Aug 06 15:54:24.957549 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: [pid: 599310|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 15:54:24 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:54:24.970368 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: Traceback (most recent call last): Aug 06 15:54:24.970368 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: from cinder.wsgi.wsgi import initialize_application Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: from cinder.common import config Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: from cinder import policies Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: from cinder.policies import attachments Aug 06 15:54:24.971047 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 15:54:24.971821 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: from cinder.policies import base Aug 06 15:54:24.971821 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 15:54:24.971821 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 15:54:24.971821 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 15:54:24.971821 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 15:54:24.971821 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: *** no app loaded. going in full dynamic mode *** Aug 06 15:54:28.895705 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: --- no python application found, check your startup logs for errors --- Aug 06 15:54:28.895705 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: [pid: 599310|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 15:54:28 2021] POST /volume/v3/55424bfdda004a01834e4deb95b6d98f/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 15:55:14.002840 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: Fri Aug 6 15:55:14 2021 - graceful shutdown triggered... Aug 06 15:55:14.003501 dv-cinder-803748-1-iscsi devstack@c-api.service[599310]: Gracefully killing worker 1 (pid: 599310)... Aug 06 15:55:14.003992 dv-cinder-803748-1-iscsi devstack@c-api.service[599311]: Gracefully killing worker 2 (pid: 599311)... Aug 06 15:55:14.005152 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 15:55:15.003350 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: worker 1 buried after 1 seconds Aug 06 15:55:15.003870 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: worker 2 buried after 1 seconds Aug 06 15:55:15.003870 dv-cinder-803748-1-iscsi devstack@c-api.service[599305]: goodbye to uWSGI. Aug 06 15:55:15.005014 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 15:55:15.005564 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 16:06:51.194054 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 16:06:51.216654 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 16:06:51 2021] *** Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: nodename: dv-cinder-803748-1-iscsi Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: machine: x86_64 Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: clock source: unix Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: pcre jit disabled Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: detected number of CPU cores: 4 Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: current working directory: / Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: detected binary path: /usr/bin/uwsgi-core Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: your processes number limit is 47680 Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: your memory page size is 4096 bytes Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: detected max file descriptor number: 2048 Aug 06 16:06:51.221109 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: lock engine: pthread robust mutexes Aug 06 16:06:51.223426 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: thunder lock: enabled Aug 06 16:06:51.223426 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 16:06:51.223426 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 16:06:51.285550 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: Python main interpreter initialized at 0x564fadb619d0 Aug 06 16:06:51.285550 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: python threads support enabled Aug 06 16:06:51.285550 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: your server socket listen backlog is limited to 100 connections Aug 06 16:06:51.285550 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: your mercy for graceful operations on workers is 90 seconds Aug 06 16:06:51.286271 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 16:06:51.286415 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: *** Operational MODE: preforking *** Aug 06 16:06:51.286783 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: *** uWSGI is running in multiple interpreter mode *** Aug 06 16:06:51.286783 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: spawned uWSGI master process (pid: 659743) Aug 06 16:06:51.290391 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: spawned uWSGI worker 1 (pid: 659757, cores: 1) Aug 06 16:06:51.290391 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: spawned uWSGI worker 2 (pid: 659758, cores: 1) Aug 06 16:06:51.290391 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 16:06:51.287047 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 16:06:53.247860 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: Traceback (most recent call last): Aug 06 16:06:53.247860 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: from cinder.wsgi.wsgi import initialize_application Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: from cinder.common import config Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: from cinder import policies Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: from cinder.policies import attachments Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: from cinder.policies import base Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: *** no app loaded. going in full dynamic mode *** Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: --- no python application found, check your startup logs for errors --- Aug 06 16:06:53.248997 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: [pid: 659758|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 16:06:53 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 16:06:53.740074 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: Traceback (most recent call last): Aug 06 16:06:53.740074 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 16:06:53.740792 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: from cinder.wsgi.wsgi import initialize_application Aug 06 16:06:53.740792 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 16:06:53.740792 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: from cinder.common import config Aug 06 16:06:53.740792 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 16:06:53.740792 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 16:06:53.740792 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 16:06:53.741297 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: from cinder import policies Aug 06 16:06:53.741297 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 16:06:53.741297 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: from cinder.policies import attachments Aug 06 16:06:53.741297 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 16:06:53.741614 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: from cinder.policies import base Aug 06 16:06:53.741614 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 16:06:53.741614 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 16:06:53.741937 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 16:06:53.741937 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 16:06:53.741937 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: *** no app loaded. going in full dynamic mode *** Aug 06 16:06:58.470695 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: --- no python application found, check your startup logs for errors --- Aug 06 16:06:58.470695 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: [pid: 659758|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 16:06:58 2021] POST /volume/v3/461abcfdd4264aaab581c3eaae115e9d/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 16:07:39.947694 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 16:07:39.948157 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: Fri Aug 6 16:07:39 2021 - graceful shutdown triggered... Aug 06 16:07:39.948638 dv-cinder-803748-1-iscsi devstack@c-api.service[659757]: Gracefully killing worker 1 (pid: 659757)... Aug 06 16:07:39.949030 dv-cinder-803748-1-iscsi devstack@c-api.service[659758]: Gracefully killing worker 2 (pid: 659758)... Aug 06 16:07:40.949392 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: worker 1 buried after 1 seconds Aug 06 16:07:40.949392 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: worker 2 buried after 1 seconds Aug 06 16:07:40.949392 dv-cinder-803748-1-iscsi devstack@c-api.service[659743]: goodbye to uWSGI. Aug 06 16:07:40.950898 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 16:07:40.951456 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 16:19:21.036494 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 16:19:21.060791 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 16:19:21 2021] *** Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: nodename: dv-cinder-803748-1-iscsi Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: machine: x86_64 Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: clock source: unix Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: pcre jit disabled Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: detected number of CPU cores: 4 Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: current working directory: / Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: detected binary path: /usr/bin/uwsgi-core Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: your processes number limit is 47680 Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: your memory page size is 4096 bytes Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: detected max file descriptor number: 2048 Aug 06 16:19:21.063344 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: lock engine: pthread robust mutexes Aug 06 16:19:21.065085 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: thunder lock: enabled Aug 06 16:19:21.065085 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 16:19:21.065085 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 16:19:21.119461 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: Python main interpreter initialized at 0x5570d54b19d0 Aug 06 16:19:21.119461 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: python threads support enabled Aug 06 16:19:21.119461 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: your server socket listen backlog is limited to 100 connections Aug 06 16:19:21.119461 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: your mercy for graceful operations on workers is 90 seconds Aug 06 16:19:21.119987 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 16:19:21.120132 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: *** Operational MODE: preforking *** Aug 06 16:19:21.120590 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: *** uWSGI is running in multiple interpreter mode *** Aug 06 16:19:21.120590 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: spawned uWSGI master process (pid: 719840) Aug 06 16:19:21.120889 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 16:19:21.124880 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: spawned uWSGI worker 1 (pid: 719849, cores: 1) Aug 06 16:19:21.125295 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: spawned uWSGI worker 2 (pid: 719850, cores: 1) Aug 06 16:19:21.125644 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 16:19:23.100686 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: Traceback (most recent call last): Aug 06 16:19:23.100686 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: from cinder.wsgi.wsgi import initialize_application Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: from cinder.common import config Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: from cinder import policies Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: from cinder.policies import attachments Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: from cinder.policies import base Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: *** no app loaded. going in full dynamic mode *** Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: --- no python application found, check your startup logs for errors --- Aug 06 16:19:23.101772 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: [pid: 719850|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 16:19:23 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 16:19:23.220503 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: Traceback (most recent call last): Aug 06 16:19:23.220503 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: from cinder.wsgi.wsgi import initialize_application Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: from cinder.common import config Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: from cinder import policies Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: from cinder.policies import attachments Aug 06 16:19:23.221145 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 16:19:23.221722 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: from cinder.policies import base Aug 06 16:19:23.221722 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 16:19:23.221722 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 16:19:23.221722 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 16:19:23.221722 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 16:19:23.221722 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: *** no app loaded. going in full dynamic mode *** Aug 06 16:19:27.800473 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: --- no python application found, check your startup logs for errors --- Aug 06 16:19:27.800473 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: [pid: 719850|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 16:19:27 2021] POST /volume/v3/82f9a52d72c24e77af042df93d14a1c9/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (1 switches on core 0) Aug 06 16:20:10.256192 dv-cinder-803748-1-iscsi systemd[1]: Stopping Devstack devstack@c-api.service... Aug 06 16:20:10.256380 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: Fri Aug 6 16:20:10 2021 - graceful shutdown triggered... Aug 06 16:20:10.256817 dv-cinder-803748-1-iscsi devstack@c-api.service[719849]: Gracefully killing worker 1 (pid: 719849)... Aug 06 16:20:10.257181 dv-cinder-803748-1-iscsi devstack@c-api.service[719850]: Gracefully killing worker 2 (pid: 719850)... Aug 06 16:20:11.257405 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: worker 1 buried after 1 seconds Aug 06 16:20:11.257405 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: worker 2 buried after 1 seconds Aug 06 16:20:11.257405 dv-cinder-803748-1-iscsi devstack@c-api.service[719840]: goodbye to uWSGI. Aug 06 16:20:11.259013 dv-cinder-803748-1-iscsi systemd[1]: devstack@c-api.service: Succeeded. Aug 06 16:20:11.259546 dv-cinder-803748-1-iscsi systemd[1]: Stopped Devstack devstack@c-api.service. Aug 06 16:32:07.075340 dv-cinder-803748-1-iscsi systemd[1]: Starting Devstack devstack@c-api.service... Aug 06 16:32:07.084875 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: [uWSGI] getting INI configuration from /etc/cinder/cinder-api-uwsgi.ini Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 6 16:32:07 2021] *** Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: nodename: dv-cinder-803748-1-iscsi Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: machine: x86_64 Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: clock source: unix Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: pcre jit disabled Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: detected number of CPU cores: 4 Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: current working directory: / Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: detected binary path: /usr/bin/uwsgi-core Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: your processes number limit is 47680 Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: your memory page size is 4096 bytes Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: detected max file descriptor number: 2048 Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: lock engine: pthread robust mutexes Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: thunder lock: enabled Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/cinder-wsgi.socket fd 4 Aug 06 16:32:07.090089 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] Aug 06 16:32:07.137428 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: Python main interpreter initialized at 0x563472c719d0 Aug 06 16:32:07.137428 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: python threads support enabled Aug 06 16:32:07.137428 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: your server socket listen backlog is limited to 100 connections Aug 06 16:32:07.137428 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: your mercy for graceful operations on workers is 90 seconds Aug 06 16:32:07.138122 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 16:32:07.138122 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: *** Operational MODE: preforking *** Aug 06 16:32:07.142301 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: *** uWSGI is running in multiple interpreter mode *** Aug 06 16:32:07.142301 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: spawned uWSGI master process (pid: 780291) Aug 06 16:32:07.142301 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: spawned uWSGI worker 1 (pid: 780304, cores: 1) Aug 06 16:32:07.142301 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: spawned uWSGI worker 2 (pid: 780305, cores: 1) Aug 06 16:32:07.142301 dv-cinder-803748-1-iscsi devstack@c-api.service[780291]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 16:32:07.138420 dv-cinder-803748-1-iscsi systemd[1]: Started Devstack devstack@c-api.service. Aug 06 16:32:08.671964 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: Traceback (most recent call last): Aug 06 16:32:08.673162 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 16:32:08.673359 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: from cinder.wsgi.wsgi import initialize_application Aug 06 16:32:08.673359 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 16:32:08.673541 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: from cinder.common import config Aug 06 16:32:08.673541 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 16:32:08.673705 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 16:32:08.673705 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 16:32:08.673832 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: from cinder import policies Aug 06 16:32:08.673832 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 16:32:08.673832 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: from cinder.policies import attachments Aug 06 16:32:08.674020 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 16:32:08.674020 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: from cinder.policies import base Aug 06 16:32:08.674020 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 16:32:08.674257 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 16:32:08.674257 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 16:32:08.674257 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 16:32:08.674257 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: *** no app loaded. going in full dynamic mode *** Aug 06 16:32:08.674496 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: --- no python application found, check your startup logs for errors --- Aug 06 16:32:08.674496 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: [pid: 780304|app: -1|req: -1/1] 10.222.0.35 () {50 vars in 776 bytes} [Fri Aug 6 16:32:08 2021] GET /volume/v3 => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0) Aug 06 16:32:08.680135 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: Traceback (most recent call last): Aug 06 16:32:08.680135 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: File "/usr/local/bin/cinder-wsgi", line 6, in Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: from cinder.wsgi.wsgi import initialize_application Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: File "/opt/stack/cinder/cinder/wsgi/wsgi.py", line 27, in Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: from cinder.common import config Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: File "/opt/stack/cinder/cinder/common/config.py", line 36, in Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: from cinder.policy import DEFAULT_POLICY_FILENAME Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: File "/opt/stack/cinder/cinder/policy.py", line 27, in Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: from cinder import policies Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: File "/opt/stack/cinder/cinder/policies/__init__.py", line 18, in Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: from cinder.policies import attachments Aug 06 16:32:08.680772 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: File "/opt/stack/cinder/cinder/policies/attachments.py", line 18, in Aug 06 16:32:08.681383 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: from cinder.policies import base Aug 06 16:32:08.681383 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: File "/opt/stack/cinder/cinder/policies/base.py", line 97, in Aug 06 16:32:08.681383 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: '(' + SYSTEM_ADMIN + ') or (' + PROJECT_MEMBER + ')' Aug 06 16:32:08.681383 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: NameError: name 'SYSTEM_ADMIN' is not defined Aug 06 16:32:08.681383 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: unable to load app 0 (mountpoint='') (callable not found or import error) Aug 06 16:32:08.681383 dv-cinder-803748-1-iscsi devstack@c-api.service[780305]: *** no app loaded. going in full dynamic mode *** Aug 06 16:32:12.694541 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: --- no python application found, check your startup logs for errors --- Aug 06 16:32:12.694541 dv-cinder-803748-1-iscsi devstack@c-api.service[780304]: [pid: 780304|app: -1|req: -1/2] 10.222.0.35 () {60 vars in 1238 bytes} [Fri Aug 6 16:32:12 2021] POST /volume/v3/ab4f9373f088434a9fe17df11732dda3/types => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0)